As you have been able to read in several posts we have published, our work consists, among others, of the development of mathematical models, machine learning models and even visual computing or language understanding models.
Some recurring questions are: How will we work with this customer? What kind of deployment will we use? What about the cloud? What if they don’t have a cloud? Once we have done this work for a customer, how do we proceed to deliver it?
The answer to these questions is simple when viewed through the prism of the DevOps philosophy: collaborate!
DevOps = Collaboration
We will first make an effort not to use DevOps terminology (meet, cloud, delivery, testing, agile, deployment, machine learning, docker, Kubernetes), as this article is about DevOps culture and not about the tools and platforms that facilitate its implementation.
According to the DevOps methodology, we must follow a series of practices to ensure a reliable software delivery process:
- Develop and test in production-like environments.
- Perform deployments using repeatable and reliable processes.
- Monitor and validate operational quality.
- Amplify feedback loops.
Transitions to production are to be made under an improvement plan and deliveries will be modified in pursuit of the business.
Measuring and Planning
Let’s ground all this theory and make some decisions.
We will start from the point where we have already identified the problem to be solved and the working group has been formed. It is important to remember that the working group must always include the end user. We can call him PM, project manager or responsible user; but he must always be present and must be taken into account in every change of direction or specifications.
Let’s start by making things as easy as possible. Ideally, we prefer to use development languages that are extensively used. For example, Python, R, C++/C#, .NET or Java. These languages almost completely assure us of two things:
- The client can maintain the solution once it has been delivered.
- Facilitates integration with different data sources: databases, Excel files, cloud storage, etc.
Another feature that will help us in the process is the use of cloud computing: AWS, Azure or Google Cloud Platform. Thanks to the cloud infrastructure we can create production-like environments in which to exploit the models and save on costs by optimising the time of use. We can also run the models as a service and not depend on our infrastructure.
Experience in developing different kinds of models allows us to measure the resources needed for exploitation:
- When we talk about optimisation models, we prioritise computational capacity: not only the thread-parallelisation of the procedure but also the clock speed of the processor. If the model is deployed in the cloud, we must ensure that the machine has adequate computational capabilities.
- If the project is a machine learning model, we must balance memory usage with computational usage. This is a difficult point since model training should be done, as far as possible, on machines with high capacities separated from the operating environment. This will allow us to save on infrastructure costs.
- If the optimisation model is complex and requires a mathematical solver, the solution is not trivial. We need a customised execution environment: computational power and resolve dependencies by installing libraries and licensing for the model.
Let’s not forget something important: it is now time to look for a platform that allows the solution to be developed, tested and exploited and that is valid for everyone.
This is where the following questions usually arise:
- How many users will work with the model?
- Will it run concurrently?
- How often will the model be run?
Once the client answers these questions, at baobab solutions we know which is the best environment that will suit our clients. If you have an optimisation project in mind and you don’t know the answers, please write to us and we will give them to you.
Develop and Test
At baobab, we like to develop, even if you don’t believe it, we like it even more than having long meetings. By default and for all projects we use a cloud platform for code versioning. The use of this technology is the basis of collaboration in the development of the project and the first stone on which we can build our collaborative environment.
The code is the result of project knowledge. It is the solution that we provide to our clients’ businesses. At baobab, we have enough experience to know that the end user must be able to access it easily and interpret it, audit it and visualise the continuous improvements that are being made at all times.
If we use Git we will know when, how and why the code works. Ideally also to see where it fails. But Git will open the door to continuous software delivery and automatic deployment.
If a client asks us: Which platform do you prefer to collaborate with during the project? Our answer is any platform that allows us to collaborate from planning to production deployment.
The above response is inherent to the platform for testing. The optimisation or forecasting model will be tested in the appropriate environment before deployment. At baobab, we make sure that we deliver after an exhaustive testing plan. The end user is always involved in this process and is responsible for analysing the data and authorising the move to production.
Deployment in Production, Fingers Crossed
In the old days, we could have exposed an SFTP to download a folder containing the instruction manual and the model ordered. Or we could have burned a CD with a silkscreen. Something like “only works with Windows 98/NT” takes that! I did my job, now you’re on your own. Fortunately, we are not at that stage of computing and together we have built a very promising future.
We now have the tools to make the delivery process simple and, above all, something we can share with our customers and check together that the solution works. This is one of the most important points in the DevOps philosophy: involving the user during all phases of the project. The delivery should start at the requirements gathering stage.
During the development phase, we will have created environments similar to the production environment. And they will also be described in the repository: infrastructure as code. We prefer to use container languages. We prefer this because it encapsulates the execution environment of the optimisation model and eliminates the unknowns of each machine where the code is executed. However, we can use alternatives such as building permanent images or declarative language to deploy in the cloud.
Declaring the infrastructure as code has several advantages:
- Repeatable deployments.
- Granular security.
- Independence from the infrastructure provider.
Question from the client: Will you accompany us in the continuous deployment processes? Answer: With pleasure.
Supervise and…Optimise!
Our work is not just code, it is the knowledge and experience of dealing with such a mathematical model. How do we deliver this? Answer: on paper, right? Today’s manuals are not delivered on paper. We use tools that are accessible and allow, once again, collaboration: GitHub/Bitbucket, Confluence, Sharepoint / Teams, Google Docs, etc.
During development, the relevant information for monitoring the application/solution will be printed on the console. We will have to review the logs and we will be able to compare the different solutions provided by the model. If we have all joined together in the planning and development stage, we will be able to locate the corrections that need to be implemented quickly.
We are pleased to see that the solution provided is transformed into benefits for the customer, whether in terms of time, return on investment or availability of resources (they almost always go hand in hand).
Production deployment is followed by a post-launch support phase in which we go hand in hand with the customer to identify improvements, bugs and the best ways to implement the delivered solutions.
We will end this post with our greatest wish and that is to continue accompanying our clients (present and future) to evolve together in the search for the best possible solutions to our as-yet-unknown problems.
