From idea to market in less than 6 months

Hi,

This is my presentation on GCP Next 2016. In this presentation I talk about how CI&T created a new product using Google Cloud Platform in less than 6 months. I explain how we got the vision, the challenges we went through and how we made every decision based on business needs. We are using Google App Engine, Compute Engine, Pub-sub and several other GCP products to build Smart Canvas.

Hope you like it!

Cheers,
Daniel V.

PaaS / IaaS: Hybrid environments FTW!

I’m so happy right now! Google just announced on March 25th the “Managed VMs” (http://goo.gl/yv1w6F), confirming what I truly believe and have been saying for a while: PaaS is not dead and it should be a very important piece on your architectural strategy. The key aspect to consider here is that the platform will NOT address all your needs, but instead of discarding it because of that, you should design a hybrid cloud solution, using both PaaS and IaaS.

See: There is nothing wrong with PaaS (http://goo.gl/NWrBmf)

The general guidelines for this approach are:

1) Use PaaS as much as you can. Ideally, it should handle all the HTTP requests coming from end-users, keeping the IaaS usage for backend workloads. Remember: you definitely want to take advantage of the effortless scalability provided by the platform.

2) If possible, try to create a MVP (minimum viable product) that can be delivered using only the capabilities provided by the platform. Doing that, you will have a much better time-to-market, lower your risks and costs.

3) Keep in mind that you will need to use the flexibility of the infrastructure at some point. That’s fine! Design a mechanism to allow your servers to communicate with the platform. Task queues are a great choice! By the way, thank you Google for solving this problem for us.

4) Communicate asynchronously as much as you can.

5) Just one more thing: before jumping into IaaS to solve a specific problem, be sure that there are no other companies offering this capability as a service. For example: you don’t need to develop a video encoding feature on the top of your favorite Linux flavor just because the platform doesn’t provide such service to you. There are several great video encoding services out there you can use.

I look forward to testing this Managed VMs, really excited about it. Long live PaaS, IaaS and Hybrid environments!

Cheers, Daniel Viveiros

Building API ecosystems on top of a stable cloud environment

Lately I’ve been exploring the future of smart computing (social, mobile, and predictive models) and how they relate to enterprise cloud.  In one recent article, I describe why “It’s Time for Enterprise Cloud to Enable the Amazing.” For the enterprise world, cloud computing can create differentiation and promote innovation inside the company. In a second article, I offer “Guidance for Combining Cloud and Social to Enable the Amazing,” providing  insight into why the most important benefit of cloud technology is the ability to move your business to where it has never been before. In this article, I will focus on a final critical component for success: why enterprise businesses must consider building APIs on top of a stable cloud environment to enable these transformative new use cases.

After years of poor software reuse, complex system integration processes, lame and heavyweight protocols and bad service-oriented architecture “patterns,” I think we have finally found an effective way to overcome these problems: APIs. As defined by Kleber Bacili (Sensedia CEO), an API (Application Programming Interface) is “data or business services exposed in the web to other applications. It’s the most popular approach used to allow data exchange between connected devices.” That’s true. This approach completely reinvented Service Oriented Architecture (SOA – and actually, I don’t even like to use this acronym anymore) and is responsible for much of the success of companies like Google, Apple, Twitter and Facebook. Today, these companies handle billions of API requests every single month, and most of the apps we love exist only because of them.

That said, businesses looking to stay competitive in the rapidly changing digital age need to keep two things in mind:

[1] There is absolutely no way for enterprise business to enable these amazing new use cases combining all the critical components of smart computing that will transform their businesses without APIs.

[2] It’s pointless to have an API strategy without a reliable cloud infrastructure strategy supporting it.

Regarding the first statement, using APIs is the best, the easiest, and by far the most effective way to foster innovation. Actually, you can not only gain agility but also outsource innovation. Outsourcing innovation means that by exposing services as APIs, you allow other developers or companies to create innovative software that integrates with your platform. Consider, for example several products and apps that integrate with Facebook or Twitter today. They are only possible because both social networks have several APIs that allow developers to create this ecosystem. Creating a system or application is good, but creating an ecosystem that allows partners, channels, and the developer community to contribute and integrate with your business is amazing. Huge companies such as eBay, Amazon and all social networks have already realized that and they are doing prospering from the new ecosystem. Retail stores, insurance companies and even traditional businesses like banks are planning and executing their roadmap to take advantage of this connected world. By the way, have you ever heard about the “Internet of Things” and “Machine-to-Machine (M2M)?” Although they sound like buzzwords right now, they will undoubtedly play a significant role in our lives in the near future. And guess what: APIs are the foundation that are making these connections and communications happen.

So now, how does cloud computing fit into this scenario? Well, here is a list of benefits and tools that explain why it’s crucial to consider a cloud platform for your API strategy:

  • Scalability: if you are going to create an ecosystem or expose your APIs to the external world, you must be prepared to scale on-demand. The few requests you have right now may become thousands or millions in the next few days. I really like the idea of using Platform-as-a-Service (PaaS) to host the services because it’s more reliable, easier to maintain and less expensive than Infrastructure-as-a-Service (IaaS).

  • Globalization: make sure that you can handle API requests as close, geographically, to the client as possible to avoid latency issues. World-class cloud providers like Google can help address this challenge.

  • Accelerators: there are some really nice tools and libraries that help you to generate APIs and clients that can be used by many devices and programming languages. One of the most exciting and promising technologies in that sense is Google Cloud Endpoints, which simplifies backend services creation for web and mobile clients.

  • Services Composition:  cloud computing can help your API strategy not only in the sense of creating new services, but also by providing you with several different ones that you can use to enhance your application. That includes: geo-localization, language translation/detection, sentiment analysis, authentication, social listening and a variety of other APIs offered by the most popular cloud vendors that are available for your company to use.

For Enterprise businesses to survive the digital disruption, they need to be where their users are, they need to allow them to use the device they want to, and they need to profoundly understand their needs to provide them with the best service possible. In this journey to “enable the amazing” new use cases and services that can transform enterprise businesses, connecting the dots is key to reaching new levels of integration, automation and innovation. Readily available APIs built on top of a stable cloud computing environment are the most important components to make this happen. It has been said, there is no social or analytics without cloud, and there is no mobile without APIs. Well I say, most importantly, there is no transformation without both cloud and APIs.

There is nothing going wrong with PaaS!

I just read the InfoQ article “What is going on with PaaS?“. In a nutshell, it says that PaaS is being adopted in a very slow pace and enumerates the reasons why it  is “failing” or at least losing space to IaaS vendors. The main reasons, according to the author, are: Confusing marketing message, Lack of maturity and Limitations. Although I understand his main points, I have a different point of view and I disagree with some arguments there.

First, I want to state the obvious: PaaS is a platform. Someday, I will write a post entitled “The Platform Dilemma”. Every time I see developers and/or managers dealing with platforms (and I’m not just talking about cloud platforms, but software platforms as well), it’s always the same old story: “it’s limited“, “we bought it to be more productive, but it’s working otherwise“, “it’s not flexible“, “we don’t know how to work with that!“, “it should be helping me but it’s driving me crazy!” etc. That happens when .Net developers start working with Sharepoint, PHP developers with Drupal and also when traditional computer engineers move to PaaS environments, like Google App Engine or Force.com.

The point here is: adopting a platform is always a tradeoff. You give up on some flexibility to get other benefits, most of the time productivity, time-to-market or lower costs. PaaS falls into that dilemma. You should not use it in 100% of your applications. You may even create hybrid architectures taking advantages of both worlds, together with IaaS. That said, in my opinion, PaaS will always have limitations and IaaS will – at least in the next few years – grow in a faster pace than PaaS. And it’s completely normal! It doesn’t mean that there is something wrong with PaaS. Is there anything wrong with Sharepoint or Drupal because we have many more projects using .Net or PHP than the related platforms?

Regard the “confusing marketing message”, I actually agree with that but I don’t blame PaaS for that, but the messy buzzword “cloud computing” in general. And again, I think it’s totally normal that the enterprise world gets a better understanding on what is IaaS (“a regular server in the cloud, if it works here in our datacenter, it’s going to work there”) than what is PaaS. And it also looks less risky jumping to the cloud by just moving from physical servers or traditional data centers to equivalent architectures inside AWS, for example. As I said before, there is nothing going wrong with PaaS, it takes time to shift an enterprise mindset and culture towards a more disruptive approach.

Finally, as a software architect, I see “cloud components” that can be assembled into a great architecture to build amazing things. That includes scalable runtime environments (aka PaaS), managed hadoop clusters, replicated database (SQL or No-SQL), highly scalable data warehouses and so on, everything in the cloud. The menu is open and I want to be free to choose the ingredients better suited to my challenge. It may sounds complex (and it is), but the final decision of which components I should use to tackle both functional and non-functional requirements must be made to deliver business value. Is it IaaS? Is it PaaS? Is it bla-bla-bla-aaS? Doesn’t matter! You know what? It’s probably going to be a best-of-breed of all the above.

Guidance for Combining Cloud and Social to Enable the Amazing

After a few years talking to several people inside and outside my company about the benefits of cloud computing, I’m truly convinced that the most important benefit is not cost reduction or the ability to have new servers up and running in a few minutes. Of course those are great capabilities that a real cloud computing environment can offer to you. But the most interesting goal you should pursue when using this approach is to use technology to move your business to where it has never been before, and do amazing things you couldn’t even consider doing a few years ago.

One use case I’m happy to see the market (including large enterprises) explore more and more is the enhanced customer engagement and experience achieved through a social networks + cloud combination. Although almost everybody understands how powerful this combo can be, very few are succeeding to really unleash the power of this synergy. Most companies are just scratching the surface in this area. In this article, I want to explore some ideas and useful guidance for those who want to dive into these waters.

1) Be smart, consider PaaS, avoid IaaS: when listening to social networks such as Facebook, Twitter and Instagram, you should be prepared to receive and process massive amounts of data. Do you want to design a physical architecture (load balancing, fail-over, auto-scale) to handle all the data or do you want to let companies like Google do that for you? What about private cloud? Well… I don’t even consider private cloud to be part of cloud computing. High upfront costs, limited scalability and no managed services to simplify or empower your solution. It’s more like a “virtualization on steroids” than cloud computing itself!

2) Be nice, quotas are limited: you may have an amazing idea to engage your customer but keep in mind that all social networks have limited usage quotas. Sometimes a percentage of the daily traffic (1%, for example), sometimes number of requests per hour (something like 5,000 calls per hour).

3) Be ethical, avoid spam and things like that: even restricted to your quotas, you must be aware that social networks don’t want to bother their users because… well, they actually need them to survive. So, forget about commenting on all posts in a timeline advertising your great new product or service. Your account may be suspended.

4) Be reserved, read more than you write: if you are planning just to read from the social networks, ok. But, if you are planning to write comments or posts in the same volume, you are going to be in trouble. Writing quotas are significantly lower than reading ones. You are going to be blocked unless you are whitelisted. Sometimes you won’t even have access to write APIs, unless you explain very well what your plans are.

5) Be nimble, things will change: you have just spent a lot of time using social networks’ APIs and everything is working like a charm right now. Great! But these same APIs might not work well tomorrow. Sometimes you will be notified upfront so you will have time to accommodate changes. Sometimes, you won’t. Consider using third party APIs. There are several companies specialized in dealing with this problem and they can make it transparent for you.

6) Be aware, social networking companies are not like IBM or Microsoft: if you work for a huge global company and are used to having heavy negotiations with your vendors to change their products to fit to your needs, think twice before taking that for granted with companies like Facebook or Twitter. First, they run a very complex computational grid, we have an important technical aspect to consider here. Second, they are the “millennials” of the service providers. They don’t put commercial relationship and politics ahead of technology. They also need scale and they are not ready (or perhaps it’s not interesting for their business model) to dispend a lot of attention just to make you happy. Need more quota? Need to be whitelisted? Start right now because it won’t be easy… or possible.

All that said and despite the pitfalls, I don’t think anyone can or should close their eyes to this new world. Yesterday’s hot sites are not hot anymore. People are not supposed to find your company or content anymore… you should engage them where they are, and where they are spending a huge amount of time talking to their “friends”, posting photos and videos and checking-in on social networks. Most of the time using their mobile devices, not desktops.

We need to get ready for a future where brands serve their customers best by anticipating their next move and by being ready to help exactly where they are in that moment of need. This will be the general expectation of all users. Combining social plus cloud can create that layer of intelligence never before possible. Going further, leveraging not only cloud and social, but also mobile and analytics is a key aspect to deliver such capabilities. Are you ready for this new technology landscape?

It’s Time for Enterprise Cloud to Enable “The Amazing”

By now, it’s obvious that investing in a cloud strategy can put companies in a different competitive space. Today, the transformative power of cloud has been realized by many — it offers a cost effective, flexible, scalable, and reliable alternative for infrastructure. We know cloud helps companies improve customer satisfaction and maintain happier employees who can work efficiently from any location with an internet connection. However, what enterprise companies are not yet considering is the potential posed by cloud to create amazing new architectures and totally new use cases they previously would have only dreamed possible. In the past for enterprise, upfront costs would have made certain ventures prohibitive – or at least too risky, especially considering internal battles for budget.

With cloud, new businesses are not only forming but also disrupting the status quo across all industries. Netflix is the go-to example of this phenomenon. Netflix completely disrupted Blockbuster’s business, not to mention the entire video rental industry. Other examples are Mailbox, Instagram, Foursquare, and Tumblr – these companies grew on the top of cloud platforms and are now worth millions or billions of dollars. The question becomes: why are these new companies achieving such great success using cloud while large enterprise companies remain reticent to adopt and use cloud to transform their business?

For the enterprise world, cloud computing can create differentiation and promote innovation inside your company. There comes a time for enterprise companies to move beyond their previous success and develop future growth from new lines of business.  How can you focus on innovation and build amazing things if you spend all our focus and energy just keeping the lights on? Using cloud, enterprise companies can immediately focus on innovation and not worry about headaches like buying thousands of servers, setting them up, configuring security, etc.  Let’s explore the key advantages of cloud and what it can enable enterprise companies to do as never before, especially in the content management and sharing arena:

Right content to the right people

Nowadays, it’s quite common to hear the challenge: “how can we deliver the right content to the right people.”  In fact, everybody knows that data is increasing, and exponentially, outgrowing our ability to process and understand all the information out there. This is making a huge impact on businesses like marketing and publishing. To stay competitive, several companies understand that they need to put in place a user-centric approach, truly understanding who is their target and being able to laser-focus messages to this audience. In order to understand the user journey, you have to consider gathering data from several different datasources (internal, external, public, private, social etc), then immediately crunch and analyze all this data and be able to answer real-time questions in a few seconds. For example: which specific content makes your potential customer buy more of your products based on what you know from him and from other people with a similar profile? What can you do automatically to accelerate a deal or to prevent churn? What is the best approach to use an e-Detailing mobile app to close more deals when visiting customers, based on previous feedback? Does cloud computing ring a bell to you here? Well, it does for innovative companies who are deploying predictive and prescriptive analytics to improve their brand awareness and content services so they can deliver what their customers really want.

Globalization

When you use cloud, you have access to many data centers around the world, from California to Japan. For example, Amazon has two data centers on the West Coast, one on the East Coast, one in Europe, one in Japan, and one in Australia. An enterprise company using Google or Amazon Cloud Services can easily deliver content worldwide using all data centers without any expectation of delays.  Let’s consider a user in Australia.  The nearest data center will host the content for him, offering optimal performance and response time, as compared to the potential for lag if the data center were in another part of the world.

The Content Ecosystem

Cloud platforms are creating a new ecosystem of information that can be easily shared among all the channels and social networks using ready-to-use services and APIs. Those who can work effectively inside this ecosystem and figure out how to optimize these channels, will definitely prevail against their competitors. And it’s not only about posting content on social networks to be relevant, up-to-date, and reliable. It’s about understanding how your customer consumes this information to then provide them a great user experience taking into consideration several variables such as devices, subjects, sources, on-line vs off-line reading, sharing, etc. Again, enterprise companies are just scratching the surface here. Startups and companies such as Google and Facebook are dictating the rules in this game.

Bottom-line: you can create really amazing new things using all those services that are available in the cloud, just putting them together, creating an API-based architecture and taking advantage of this content ecosystem.

Now, more and more startups are using cloud to create entirely new business models. If they weren’t perceived as competition to enterprise before they are now simply because they are able to take risks. Netflix disrupted Blockbuster. Spotify is putting the iTunes store in jeopardy. Not all risks pay off but new start ups are there and can can quickly interrupt an enterprise business model. If smaller start ups couldn’t pose a threat to enterprise businesses, large companies could just stay in their comfort zone without taking risks. Now everyone has to take risks just to stay competitive.

The time has come for enterprise to use cloud to really transform how we work and live. Geoffrey Moore’s book Escape Velocity explored large companies’ gravitational power compared to smaller companies without this pull. Without this force working against them, the smaller companies could fly. Similarly, it’s time for enterprise to start thinking about how to use cloud as an escape velocity towards their own innovation.