Orchestration and Workflow
Orchestrator is a software that manages the interconnections and interactions of various processes running across heterogeneous systems in multiple locations.
Cloud Orchestration involves co-ordinated management of Compute, Storage and Network across heterogeneous systems in single or multiple locations. There are myriad of Cloud Orchestration products available today such as Openstack, vCloud, Windows Azure Pack, Cloudstack, Eucalyptus, OnApp, etc.
Service Orchestration process manages the interconnections and interactions of various tools that cross all management disciplines and interact with all types of infrastructure process such as provisioning, security, monitoring, backup, logging, compliance and application process such as Configuration Management, Application Performance and databases.
There are various provisioning, automation and configuration management tools such as Puppet, Chef, Saltstack, Ansible, etc. available today, but they are not designed for service orchestration. Ticketing tools normally tend to fill-in that gap which these tools do not address.
In Service Orchestration, the ‘how’ to co-ordinate between the various system gets specified using the tools. Quite often the ‘who’ and ‘when’ parts are not given the due consideration. Workflow based Service Orchestration can help in managing the roles and users to the various orchestration tasks in the service orchestration process.
Why workflows for Service orchestration?
In many cases where the problem space is relatively simple, orchestration could possibly be achieved with a bunch of scripts (could be vanilla shell, chef, puppet, salt, ansible, etc.). But as systems become more complex, the script would become unwieldy. For eg. additional logic might be required to make sure several processes across different apps/tools had completed before starting the dependent process. Also there could be time bound constraints that mandate that a task cannot be started prior to some boundary event or timeline. These situations would require that experts in specific areas who work on tasks that constitute the process collaborate effectively to ensure the service orchestration happens correctly.
Today’s systems are more complex and invariably require specialists in each area to oversee them. We see rapid evolution of technologies in the cloud arena. There is a profusion of tools, right from numerous Cloud platforms, Infrastructure Monitoring, Application Monitoring, Process Monitoring, Security, Logging, Deployment, Configuration Management systems. It would be quite difficult to write modular scripts and maintain their evolution in a cogent fashion across these specialized teams. Most organizations today manage with a ticketing tool, where the service request gets routed through various teams and each team uses its own scripts/tools to finish their part of the work before the service eventually gets delivered. The limitations with this solution are:
1. Many tasks get done via a single ticket, that it becomes difficult to infer the status, bottlenecks, time-spent by different teams for their portion of the work.
2. There cannot be a time bound constraint specified for a task in the process even if the situation demands that such be imposed
3. For a new member in any team, it is not intuitive to infer the downstream dependents who are waiting for the output of his/her task in order for them to start theirs
All of the above contribute to the responsiveness or turn-around time for service delivery. Businesses today require agility that necessitates their new services be made available in the shortest possible time. We realize that service orchestration needs coordination among different roles to get the services up and running. A workflow system brings precisely this functionality of ‘when’, ‘who’ needs to do the ‘what’. Having a workflow model defined for the service process orchestration helps in improving the productivity and agility.
Typically in a SME, some of the above mentioned needs might not be there or not so pressing. Consequently their information systems landscape could be easier to manage as compared to a large enterprise. However, there will always be the new technologies emerging on the horizon that promise better functionality or better performance which could be a potential differentiator in the efficiency of the system. So both SMEs and large enterprises will experience the need to engage specialists to deal with the technology changes arising on the horizon. Quite often, organizations are able to adapt to these changes and survive and a few even thrive. Such organizations invariably have systems in place with clearly defined roles and responsibilities. This helps their employees to understand clearly what skills they should learn/upgrade or the teams hire specialists to manage the new technology or tool.
What functionality do the workflow engines provide?
Workflow engines provide a facility to define a workflow process model and execute instances of the model to generate tasks. A process model consists of a series of tasks and events from the start of the process to its various possible termination points. The tasks could be assigned to a user or a group of users and also flagged with a due date or start date. It is through these functionalities that the ‘who’ and ‘when’ part of service orchestration can be defined. Additionally some workflow engines provide a mechanism to share runtime state between tasks involved in that process instance.
All the modelling information is represented in XML. The workflow domain is mature and the industry has a standard way of representing workflows, currently it is the BPMN2.0 standard. Any modelling tool that can generate BPMN2.0 compliant xml can be used to define the workflow process models. Any BPMN2.0 compliant workflow engine will be able to parse such xml definitions and accordingly create the subsequent workflow tasks.
Below is a sample workflow process for a distributed application deployment lifecycle.
Each workflow process model execution could vary based on the inputs given at run time to the various tasks that constitute the workflow process. In fact the process models usually have logic defined in them which decides the path of the execution based on these input values to their constituent tasks.
In the next part of this blog topic, we will see how exactly service orchestration is married to workflow in CORESTACK.