GCP: qual abordagem de armazenamento devo usar?

Trabalhando em clientes que usam Google Cloud Platform (GCP), uma das perguntas mais frequentes que ouço e participo de discussões é: qual a melhor tecnologia de armazenamento para utilizar em determinada situação? A plataforma oferece várias e muitas vezes é complicado tomar essa decisão:

  1. Datastore
  2. GCS
  3. Firebase (realtime database e storage)
  4. Spanner
  5. BigQuery
  6. CloudSQL
  7. Big Table
  8. Qq coisa rodando no GCE
  9. Etc…

E aí? A Google essa semana fez um favor de dar um guia para isso que acho bastante útil. Ela publicou essa foto aqui.

Screen Shot 2017-06-05 at 5.11.39 PM

Não precisa ser levado a ferro e fogo, mas certamente dá uma direção para saber o que usar e – principalmente – o que não usar. Por exemplo: lendo o diagrama acima, vc acha uma boa idéia usar o BigQuery para workloads não-analiticos? Ou o Spanner para analíticos? Não é impossível, mas ao trilhar esse caminho saiba que vocês está nadando contra a maré, cuidado.

Abraços!

IaaS é commodity. Qual o real benefício da nuvem?

No dia 16/06 aconteceu o Google Next – evento sobre Google Cloud Platform (GCP) – em várias cidades como Nova Iorque, Tokyo e São Francisco com live stream para o mundo inteiro. O keynote foi bem interessante e me fez refletir sobre o ponto que estamos no assunto cloud computing e o que esperar do futuro próximo. Compartilho abaixo algumas das  minhas reflexões:

1. Acabou o medo de se adotar infraestruturas e plataformas em nuvem

Pelo menos para cenários como hospedagem de aplicações não-críticas, campanhas de marketing, big data ou aplicações digital / mobile. Há mais de 5 anos trabalhando quase que exclusivamente com o assunto, é nítida para mim a diferença de comportamento e aceitação de empresas de todos os tamanhos com o assunto. Hoje eu vejo empresas de todos os tamanhos e nacionalidades tratando o assunto com total naturalidade. A exceção acontece em alguns poucos segmentos mais conservadores ou fortemente regulados, mas mesmo assim movimentos significativos já acontecem em setores financeiros e farmacêuticos, por exemplo.

2. IaaS está se tornando commodity

Principalmente se considerarmos os principais provedores mundiais de nuvem. E commodity no sentido mais puro da palavra: pouca diferenciação por valor agregado e maior diferenciação por preço. Apesar do entusiasmo das empresas ao moverem os seus primeiros sistemas para um servidor rodando na nuvem, elas estão apenas no primeiro passo da jornada. Cada dia mais, a decisão de adotar GCP, AWS ou Azure – para IaaS – se parece com a decisão de ter um chip Vivo, Claro ou TIM. Algumas diferenças aqui e ali em termos de funcionalidades, mas condições comerciais e preços acabam sendo os principais fatores de decisão.

3. A diferenciação em potência de 10 acontece com serviços gerenciados

O salto de 10x de agilidade e agressividade de posicionamento digital vai acontecer para os que souberem fazer o melhor uso dos serviços gerenciados em nuvem. São eles que vão permitir criar coisas incríveis como aplicações com escalabilidade infinita em semanas, analisar e gerar insights a partir de volumes estrondosos de dados, manter sincronização real-time entre dispositivos etc, tudo isso com foco apenas em desenvolvimento. Isso significa colocar o foco e energia da sua empresa no local correto: gerar valor. E não em complexas arquiteturas lógicas e de infraestrutura. E aqui teremos um dilema que as empresas vão precisar lidar: vendor lock-in. Serão 3 opções:

IaaS para ter total controle com menor produtividade de desenvolvimento e maior custo de gerenciamento e operações;
PaaS e Serviços Gerenciados para ter maior produtividade e agilidade, muitas vezes sem necessidade de investimento em operações, utilizando soluções e arquiteturas definidas pelo provedor (vendor lock-in);
CaaS (Containers-as-a-Service) para ficar no meio termo, conciliando controle e abstração de ambiente de execução e escalabilidade. Por enquanto, tecnologia ainda amadurecendo;

Meu palpite é que grandes empresas vão considerar as três opções e decidir a melhor sob demanda.

E a Google Cloud Platform nesse cenário?

Vejo com bastante entusiasmo a posição atual da GCP nesse cenário desenhado. O evento “NEXT” mostrou como empresas como JDA e PwC estão adotando a plataforma no mundo corporativo com sucesso. Depoimentos de executivos dessas empresas deixaram clara a consistência de performance, escalabilidade e segurança da plataforma. E o mais interessante é que a GCP possui ofertas para todos os cenários que mencionei:

IaaS com Google Compute Engine: para hosting de aplicações legadas ou novos desenvolvimentos que demandem maior controle de código e infra;
PaaS com Google App Engine: para sistemas com alta-disponibilidade e escalabilidade e foco em produtividade e desenvolvimento;
Containers com Google Container Engine e Kubernetes: para criação e gerenciamento de cluster de conteineres;
Big Data com Big Query, Cloud Storage, Dataflow, Pub-sub etc: serviços gerenciados para processar, armazenar e analisar altos volumes de dados em tempo real;
Firebase e Cloud endpoints: para criar aplicações móveis utilizando serviços em nuvem e com sincronização em tempo real de todos os dispositivos conectados;

O distanciamento com relação a concorrentes vai acontecer para quem conseguir explorar ao máximo o que as plataformas de nuvem têm a oferecer. E principalmente para quem explorar os serviços gerenciados oferecidos por estas plataformas. Como disse o executivo da PwC no evento: “_Google is the original cloud company_”. Eu concordo. Na minha visão, a Google lidera indiscutivelmente em visão e execução de entrega destes serviços.

There is nothing going wrong with PaaS!

I just read the InfoQ article “What is going on with PaaS?“. In a nutshell, it says that PaaS is being adopted in a very slow pace and enumerates the reasons why it  is “failing” or at least losing space to IaaS vendors. The main reasons, according to the author, are: Confusing marketing message, Lack of maturity and Limitations. Although I understand his main points, I have a different point of view and I disagree with some arguments there.

First, I want to state the obvious: PaaS is a platform. Someday, I will write a post entitled “The Platform Dilemma”. Every time I see developers and/or managers dealing with platforms (and I’m not just talking about cloud platforms, but software platforms as well), it’s always the same old story: “it’s limited“, “we bought it to be more productive, but it’s working otherwise“, “it’s not flexible“, “we don’t know how to work with that!“, “it should be helping me but it’s driving me crazy!” etc. That happens when .Net developers start working with Sharepoint, PHP developers with Drupal and also when traditional computer engineers move to PaaS environments, like Google App Engine or Force.com.

The point here is: adopting a platform is always a tradeoff. You give up on some flexibility to get other benefits, most of the time productivity, time-to-market or lower costs. PaaS falls into that dilemma. You should not use it in 100% of your applications. You may even create hybrid architectures taking advantages of both worlds, together with IaaS. That said, in my opinion, PaaS will always have limitations and IaaS will – at least in the next few years – grow in a faster pace than PaaS. And it’s completely normal! It doesn’t mean that there is something wrong with PaaS. Is there anything wrong with Sharepoint or Drupal because we have many more projects using .Net or PHP than the related platforms?

Regard the “confusing marketing message”, I actually agree with that but I don’t blame PaaS for that, but the messy buzzword “cloud computing” in general. And again, I think it’s totally normal that the enterprise world gets a better understanding on what is IaaS (“a regular server in the cloud, if it works here in our datacenter, it’s going to work there”) than what is PaaS. And it also looks less risky jumping to the cloud by just moving from physical servers or traditional data centers to equivalent architectures inside AWS, for example. As I said before, there is nothing going wrong with PaaS, it takes time to shift an enterprise mindset and culture towards a more disruptive approach.

Finally, as a software architect, I see “cloud components” that can be assembled into a great architecture to build amazing things. That includes scalable runtime environments (aka PaaS), managed hadoop clusters, replicated database (SQL or No-SQL), highly scalable data warehouses and so on, everything in the cloud. The menu is open and I want to be free to choose the ingredients better suited to my challenge. It may sounds complex (and it is), but the final decision of which components I should use to tackle both functional and non-functional requirements must be made to deliver business value. Is it IaaS? Is it PaaS? Is it bla-bla-bla-aaS? Doesn’t matter! You know what? It’s probably going to be a best-of-breed of all the above.

Guidance for Combining Cloud and Social to Enable the Amazing

After a few years talking to several people inside and outside my company about the benefits of cloud computing, I’m truly convinced that the most important benefit is not cost reduction or the ability to have new servers up and running in a few minutes. Of course those are great capabilities that a real cloud computing environment can offer to you. But the most interesting goal you should pursue when using this approach is to use technology to move your business to where it has never been before, and do amazing things you couldn’t even consider doing a few years ago.

One use case I’m happy to see the market (including large enterprises) explore more and more is the enhanced customer engagement and experience achieved through a social networks + cloud combination. Although almost everybody understands how powerful this combo can be, very few are succeeding to really unleash the power of this synergy. Most companies are just scratching the surface in this area. In this article, I want to explore some ideas and useful guidance for those who want to dive into these waters.

1) Be smart, consider PaaS, avoid IaaS: when listening to social networks such as Facebook, Twitter and Instagram, you should be prepared to receive and process massive amounts of data. Do you want to design a physical architecture (load balancing, fail-over, auto-scale) to handle all the data or do you want to let companies like Google do that for you? What about private cloud? Well… I don’t even consider private cloud to be part of cloud computing. High upfront costs, limited scalability and no managed services to simplify or empower your solution. It’s more like a “virtualization on steroids” than cloud computing itself!

2) Be nice, quotas are limited: you may have an amazing idea to engage your customer but keep in mind that all social networks have limited usage quotas. Sometimes a percentage of the daily traffic (1%, for example), sometimes number of requests per hour (something like 5,000 calls per hour).

3) Be ethical, avoid spam and things like that: even restricted to your quotas, you must be aware that social networks don’t want to bother their users because… well, they actually need them to survive. So, forget about commenting on all posts in a timeline advertising your great new product or service. Your account may be suspended.

4) Be reserved, read more than you write: if you are planning just to read from the social networks, ok. But, if you are planning to write comments or posts in the same volume, you are going to be in trouble. Writing quotas are significantly lower than reading ones. You are going to be blocked unless you are whitelisted. Sometimes you won’t even have access to write APIs, unless you explain very well what your plans are.

5) Be nimble, things will change: you have just spent a lot of time using social networks’ APIs and everything is working like a charm right now. Great! But these same APIs might not work well tomorrow. Sometimes you will be notified upfront so you will have time to accommodate changes. Sometimes, you won’t. Consider using third party APIs. There are several companies specialized in dealing with this problem and they can make it transparent for you.

6) Be aware, social networking companies are not like IBM or Microsoft: if you work for a huge global company and are used to having heavy negotiations with your vendors to change their products to fit to your needs, think twice before taking that for granted with companies like Facebook or Twitter. First, they run a very complex computational grid, we have an important technical aspect to consider here. Second, they are the “millennials” of the service providers. They don’t put commercial relationship and politics ahead of technology. They also need scale and they are not ready (or perhaps it’s not interesting for their business model) to dispend a lot of attention just to make you happy. Need more quota? Need to be whitelisted? Start right now because it won’t be easy… or possible.

All that said and despite the pitfalls, I don’t think anyone can or should close their eyes to this new world. Yesterday’s hot sites are not hot anymore. People are not supposed to find your company or content anymore… you should engage them where they are, and where they are spending a huge amount of time talking to their “friends”, posting photos and videos and checking-in on social networks. Most of the time using their mobile devices, not desktops.

We need to get ready for a future where brands serve their customers best by anticipating their next move and by being ready to help exactly where they are in that moment of need. This will be the general expectation of all users. Combining social plus cloud can create that layer of intelligence never before possible. Going further, leveraging not only cloud and social, but also mobile and analytics is a key aspect to deliver such capabilities. Are you ready for this new technology landscape?