Your Microservices Effort Will Fail If You Only Focus On The Technical Details

I have self-sensored stories about microservices, because I have felt the topic is as charged as linked data, REST, and some parts of the hypermedia discussion. Meaning there are many champions of the approach, who insist on telling me, and other folks how WRONG they are in the approach, as opposed to helping us work through our understanding of exactly what is microservices, and how to do well.

For me, when I come across tech layers that feel like this, they are something that is very tech saturated, with the usual cadre of tech bros leading the charge, often on behalf of a specific vendor, or specific set of vendor solutions. Even with this reality, I've read a number of very smart posts, and white papers on microservices in the last year, outlining various approaches to designing, engineering, and orchestrating your business, using the "microservices diet".

Much of what I read nails much of the technology in some very logical, and sensible ways--crafted by some people with mad skills when it comes to making sense of very large companies, and software ecosystems. Even with all of this lofty thinking, I'm seeing one common element missing in most of the microservices approaches I have digest--the human element.

I hear a lot of discussion about technical approaches to unwinding the bits and bytes of technical debt, but very little guidance for anyone on how to unwind the human aspect of all of this, acknowledging the years of business and political decisions that contributed to the technical debt. Its one thing to start breaking apart your databases, and systems, but its another thing to start decoupling how leadership invests in technology, the purchasing decisions that have been made, and the politics that surrounds these existing legacy systems.

I don't know about you, but every legacy system I've ever come across almost always has had a gatekeeper, an individual, or group of individuals who will fight you to the death to defend their budget, and what they know about tech (or do not know). I've encountered systems who have a dedicated budget, which only exists because the system is legacy, and with that gone, the money goes away too--sell me a microservices solution for this scenario!

Another dimension to this discussion, is that investors in microservice solutions are not interested in their money being used for this area. It just isn't sexy to be spending money on dealing with corporate politics, and unwinding years of piss poor business decisions, and educating and training the average business user around Internet technology. If you do not unwind these very human led, politically charged, business environments, you will never unwind the systems that exist within their tractor beams. Never. I'd care how much YOU believe.

In the end, I'm not trying to make you feel like you are going to fail. My goal is to encourage more investment in this area by the microservice pundits, vendors providing solutions the space, and VC's who are pouring money into these solutions. Many of the young, energetic folks at the helm of startups do not fully grasp the human side of corporate operations, and the potential quagmire that exists on the ground in front of them.

I am hoping that a handful of service providers out there can lower the rhetoric around their services and tooling, so that expectations get set at more realistic levels. Otherwise the push-back against the first couple failed waves of microservice implementations will become impenetrable, and blow any chance of making it work.

See The Full Blog Post

API Design, IoT, and Microservices Dominate The Talk Submissions For APIStrat Austin 2015

We closed the call for papers for APIStrat Austin 2015 last week, and have been processing the almost 100 talks that were submitted. We already had some session topics identified for the API conference this November 18th, 19th, and 20th in Austin, TX, but were also delighted to see the vast number of topics submitted through the call for papers process, resulting in this tag cloud.

This edition of APIStrat doesn't have a theme--we are just making about the roots of the conference, and looking to focus the conversation around the most pressing issues we face across the space at the end of 2015--I think this tag cloud reflects that conversation.

We have gone through all the talks, grouped them by topic, and will be reviewing them with the session chairs we have assigned to help organize and manage each track. By October, we will begin publishing the calendar, and sharing more detail about each session track, and the speakers in attendance.

While call for papers is closed, we have a handful of sponsorship opportunities left, so contact us if you want to help support the conversation at APIStrat. At the very least, make sure you are registered before the very intimate event sells out, you won't want to miss the conversations that occur.

See The Full Blog Post

Why You Are Not Doing Microservices Right!

Yeah, I know, it is a click-bait title. My goal is to not flame the haters, but educate everyone involved, both the people looking to learn about microservices, and those who are practicing microservices, so I'm hoping you will forgive me--my goal is not page views.

I'm sticking to my goal of not making microservices a regular thing I talk about, but I wanted to share some observations after listening to folks talk at Gluecon last week--a week that was full of microservices love (and hate). In the past, I mentioned that I received a handful of emails, and DMS from folks letting me know I'm not doing microservices, and after Gluecon I think I have a beginning list of why I think people generally are telling me I'm doing it wrong.

  • Tooling - People tend to identify with the tooling they use, and deeply associate the function with the tool. So if I don't use Puppet or Chef, I'm not orchestrating. If your not using Jenkins, your not doing continous delivery, and so on. This is dangerous for microservices storytelling, but also is something that will continue.
  • Vendor - This is probably the most egregious reason I see people telling others they are doing microservices wrong, and is somewhat akin to the tooling item, but is really just rooted in selling you something. You will never do it right, unless you are their customer, so don't worry about it.
  • Scope - You just aren't doing it at the scope someone else is doing it. Microservices is something large enterprises do, not small companies, or individuals. You have to be operating across many availability zones, many servers, etc. Horseshit. Microservices can be done at any scope, don't listen to them.
  • Completeness - You are only doing some of the aspects of microservices. Maybe you don't need to orchestrate across many operational zones, and your traffic looks very different than some of the usual suspects. Microservices is very seldom a complete picture, it is more about doing what works for you, cherry picking the valuable items you can apply in your world, not any complete definition.
  • Incomplete Info - Someone just read a single blog post you wrote, and hasn't been following the series, asked any questions, and just make assumptions about a single thing you wrote. While I feel you should make each post as complete as possible, the responsibility is on the reader to do their homework - Bwahahaha! yeah right, pundits will never do that.

So far these are the biggest reasons that I've extracted from people me telling me I am not doing it right. Ultimately it comes down to power and control, which I saw play out with SOA, REST, Linked Data, and other areas of "API". They either want to make you feel small, or not cool because you aren't using the tools they are using, or they just want you to buy their snake oil. 

In the end, for me, microservices are just a new storytelling package. They are mostly marketing, as many of the skeptics like to point out, and the companies who are truly doing microservices, will care about sharing their real-world stories, and work overtime to help you understand--there are a handful of them out there, like Runscope and Iron.io

For me, I will be focusing on the moving parts of microservices, which is why you'll see me talking about API design, deployment, management, testing, monitoring, security over just "microservices" umbrella. I could easily attack many of the pundits who challenge me, and say you aren't doing API management, evangelism, monetization, or other critical areas right, but why would I? Where does that get us?

As usual, if you have any questions, let me know how I can help. I'm happy to share, as I try to make sense of all of this. I'm adding some new research sites to support the conversation. 

See The Full Blog Post

On APIs and Microservices

I've been monitoring an emerging slice of the API, that has been dubbed "microservices" for some time now, and you've even heard me explore its use, when describing my architectural approach to redesigning more core API stack for my internal systems. I’m slowly redesigning my internal API stack, keeping them as small as possible, throughout all aspects of operations, deploying each endpoint in a single dockerized container, that contains the OS, database, web server and all server side API code. I'm also using Github, in conjunction with APIs.json to assist me in my orchestration, throughout every aspect of these APIs lifecycle from design to testing—all a new approach for me when managing my APIs.

Is defining my APIs in this way, so that the definition is much more than just the actual API endpoints, but also include the entire backend, and lifecycle workflow Microservices? Fuck if I know. One thing I do know, is I've had enough conversations with folks who are doing microservices, resulting in me saying several times that the micro services definition is something very personal. Something which the first 10 times I said , sounded very positive, but after experiencing multiple people telling me what I'm doing isn't microservices, I can tell the term definitely will continue to be very personal—very much in the same way conversations around REST or Hypermedia has gotten personal on forums, and HN threads.

You know what this tells me about microservices? Is it is more about power, control and influence, everything from enterprise architects stating their endorsement of the concept, down to the individual architects, who love to make sure you know that the way you are doing it, is WRONG! I don't think there is a single definition of "microservice", and much like REST, I think we'll hear plenty of “right” definitions, by the enterprise justifying what they are doing, and those who are selling something to the enterprise. Is there anything wrong with this? Nope. Just not my style, not my game, and has shown me over the couple months how the concept is not a fit for me.

I’m going to stick in the realm of API. It is a completely bullshit term, that means everything, but you know what? Somehow it escaped the IT, architect, and vendor ownership that SOA and now microservices possesses. For me APIs have long been more than just the tech, it is also about the business and politics of any platform implementation, so why can’t it contain my architecture styles as well? So from here forward I will be closely paying attention to microservices conversations, but you won’t hear me use the word, as I’m more comfortable in API land, and what it has come to mean to the wider public—I just don’t give a shit about what the enterprise adopts, or doesn't.

Microservices will reflect the power and control of the entities behind them, unlike APIs, which also possesses that characteristic, but also quickly shifts the conversation to be more about the innovation and opportunity that developers bring to the table, and the end-users who put that to use. Let’s make sure as we do not get caught up in the architectural discussions behind the technical curtain, and that we remember why all of this API stuff is working. Let's not get caught up in all the value being just about our architectural styles, and re-live a classic IT tale, ignoring that the real value is what people actually do with our services.

See The Full Blog Post

The Enterprise Will Make The Same Mistakes With API And Microservices That They Did With SOA, Because Essential API Concepts Go Right Over Their Head

I would say the enterprise space fleet has successfully shifted their course, heading in the general direction of everything API. SAP, Oracle, IBM, Microsoft, and the rest of the usual suspects have pledged their faith to APIs, mobile, and IoT, all in a very public way. For those of us who jumped off the SOA bandwagon a while back (getting on the API bandwagon, yeehaw), this can be amusing to watch, cause the API bandwagon is better y'know?. ;-)

I sincerely hope that these large companies can make a shift towards an API way of life, or a microservice devops way of operating, or whatever the cool enterprise kids are calling it these days. Some companies will find success in these areas, while many, many others will just talk about it, and buy a wide range of vendor services that will help them talk about it—there will be a lot of money to be made (and spent).

The thing I fear, is that many of the core principles of why this whole thing is working will go right over the heads of many technologies and business leaders in the enterprise. Elements like simplicity, transparency, openness, and functioning in a decoupled, decentralized way. You are the enterprise, many of the essential ingredients of API, are just fundamentally at odds with what you are. You are a big, complex, powerful, and political corporate presence—a beast that does not like being taken apart, reinvented over and over, and being forced to work in concert with other internal, partner, and public resources, in an open way.

This doesn't mean that some enterprise entities won’t be successful—some will. I will be carefully watching the waves of enterprise organizations, for these tech savvy few, who internalize the core philosophy around APIs, successfully uploading their corporate soul to the API singularity. However, I predict I will have to work very hard to find these companies amidst the wreckage of the enterprise organizations that make the same mistakes they did with SOA, because APIs are just not compatible with their DNA, and ultimately will be rejected for reasons that are more enterprise than they are API.

See The Full Blog Post

Visualizing And Exploring My Microservices Catalog Using APIs.json With Swagger.ed

I'm just getting started exploring the ways to use APIs.json when it comes organize my new Docker fueled, micro services stack. I’ve been using APIs.son to describe each micro service, as well as define the overall collection of almost 20 micro services. I’m using the include collection, as a navigation for the loosely coupled stack of micro-services, and my friend Chris Spiliotopoulos (@chefarchitect), has done some more work with his Swagger.ed, delivering some APIs.json goodness that is in alignment with where I want to take all of this.

Adding to the discovery and visualization work he did for Swagger, Chris enabled Swagger.ed to look for valid APIs.son files as well, so when you browse to any APIs.son, like the one I have for my micro services stack, you see the little APIs.son search icon in the address bar.

When you click on the APIs.son search icon in the address bar, you get a very cool visualization. Its pretty basic at the moment, just a visual catalog of the APIs available in the include collection of my stack, but when you connect up with the Swagger visualization work he's already done, we could have a pretty cool API catalog for managing and exploring microservices.

I have almost 20 micro-services listed, and Swagger.ed gives me the ability to navigate it in a very interactive way. Whats next? We don’t know…it is about exploration, and finding out the most meaningful way of exploring the APIs I deploy and aggregate into APIs.son collections.

I can see having a visual catalog of all of my API design that I collect, then deploy, and evolve them as needed for various parts of my infrastructure or for my clients, into other sub-collections? Disconnected collections? Loosely coupled collections? Not sure how tight I want things, part of the micro service definition for me is the size of the network connecting the services, as well as the services theselves.

See The Full Blog Post

API Management Infrastructure And Service Composition Is Key To Orchestration With Microservices In A Containerized World

As I work to redefine my world using microservices, I have this sudden realization how important my API management infrastructure is to all of this. Each one of my microservices are little APIs that do one thing, and do it well, relying on my API management infrastructure to know who should be accessing, and exactly how much of the resource they should have access to.

My note API shouldn’t have to know anything about my users, it is just trained to ask my API management infrastructure, if each user has proper credentials to accessing the resource, and what the service composition will allow them to do with it (aka read, write, how much, etc.) My note API does what it does best, store notes, and relies on my API management layer to do what it does best--manage access to the microservice.

This approach to API management has llowed me to deploy any number of microservices, using my API management infrastructure to compose my various service packages—this is called service composition. I employ 3Scale infrastructure for all my API / microservice management, which I use to define different service tiers like retail, wholesale, internal, and other service specific groupings. When users sign up for API access, I add them to one of the service tiers, and my API service composition layer handles the rest.

Modern API management service composition is the magic hand-waiving in my microservice orchestration, and without it, it would be much more work for me to compose using microservices in this containerized API world that is unfolding.

Disclosure: 3Scale is an API Evangelist partner.

See The Full Blog Post

What Is Missing On My Microservices Using APIs.json

I'm using APIs.json to organize my swagger defined microservices running in docker containers, and using the machine readable API index to drive navigation between microservices organized in a single collection. APIs.json provides a simple, machine readable way to index the technology, business, and political elements of each microservice I deploy.

As I was auditing the 18+ microservices I’ve setup for my core operations, I wanted to audit each one, and make sure I had included various elements in each APIs definition, like where developers can onboard, terms of service, and where to find related code samples. To accomplish this, I was able to just compare each microservice APIs.json, with a master APIs.json template I had established while planning my original microservice stack. 

Sometimes it is just as important to know what is missing, right alongside what is available for all of my microservices, and APIs.json is proving to be a simple way of understanding this layer of my API operation at scale.

See The Full Blog Post