From Monolith to Microservice: Lessons learned from building Foxsports automation platform on Mule

I recently left Foxsports after long five eventful years. Over this period, I was very fortunate to be a part of the core backend team that built the foxsports core integration platform / microservice platform from scratch that automates the workflow from controlling the video router switch to delivering realtime sports data to amazon cloud handling 70 million api calls and servicing all live & statistical sports data to news.com.au mastheads.

The heart of the framework was ScheduAll : A resource and personnel scheduling desktop app thats used widely amongst Broadcast and Network management companies to schedule WorkOrder. This is a tale of how we hooked into the thin integration layer of the system and integrated other network components of the System to give them life. How a WorkOrder booking evolved from a manual spreadsheet job order into self managing , self healing business intelligence framework that needs minimum to no human interaction. Thus transforming the traditional Broadcast infrastructure into a modern Stream based on demand system.

You can see our CTO talking more about it in this Mule Presentation.

This series of blog post will be my humble attempt to document some of the lessons learnt over the process of building a monolith and breaking it up into microservices v.1 and re-writing it again into v.2 over the period of 5 years.

I will try to focus on the following topics:

  1. The self emerging pattern of microservice from trying to do it right
  2. Api is the King – Api Management , Mule support for RAML, Api Gateway , AWS
  3. Splitting the Monolith – Shared api pattern , IPC
  4. Microservice Framework – Spring Integration vs Mule
  5. Event Driven Architecture, Customized persistent Queue Pattern  on Elasticsearch as CQRS implementation
  6. Publish-Subscribe alternative to EventBroker
  7. DataStore – To Share or Not
  8. Mule Flows vs Groovy vs Java Argument
  9. Distributed executors over HazelCast
  10. Troubleshooting – Logging , Event Monitor , Spike Detection , Right Amount of Logging, Logging Policy
  11. Microservice Deployment , Mule MMC , Containers (Docker) & Service discovery , Elastic Container Service , CloudFormation
  12. Testing – Integration vs Functional , Mocking Proxy vs Real Service, How much Unit Test is good, TDD – to do or not to do

By no way, I claim to be right or wrong. This is meant to be my Brain Dump and my thoughts in Retrospect….I am not sure how soon I can finish it though given my busy schedule in a new job and my lack of interest in writing, I have allocated myself 6 month for finishing up this full series 🙂

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s