Architecture is a minefield. And creating a good architecture for new innovative machine learning systems and applications is an unpaved road. Architecture is not by definition high level and sometimes relevant details are of the utmost importance. But getting details of the inner working on the implementation level of machine learning algorithms can be very hard. So a reference architecture on machine learning should help you in several ways.
Discussions on what a good architecture is, can be a senseless use of time. But input on this reference architecture is always welcome. This to make this reference architecture more generally useful for different domains and different industries.
The key principles that are used for the reference machine learning architecture are:
- The most important machine learning aspects must be addressed.
- The quality aspects: Security, privacy and safety require specific attention.
- The reference architecture should address all architecture building blocks from development till hosting and maintenance.
- Translation from architecture building blocks towards FOSS machine learning solution building blocks should be easily possible.
- The machine learning reference architecture is technology agnostics. The focus is on the outlining the conceptual architecture building blocks that make a machine learning architecture.
By writing down these principles is will be easier to steer discussions on this reference architecture and to improve this machine learning architecture.
Machine learning architecture principles are used to translate selected alternatives into basic ideas, standards, and guidelines for simplifying and organizing the construction, operation, and evolution of systems.
Important concerns for this machine learning reference architecture are the aspects:
- Business aspects (e.g capabilities, processes, legal aspects, risk management)
- Information aspects (data gathering and processing, data processes needed)
- Machine learning applications and frameworks needed (e.g. type of algorithm, easy of use)
- Hosting (e.g. compute, storage, network requirements but also container solutions)
- Security, privacy and safety aspects
- Maintenance (e.g. logging, version control, deployment, scheduling)
- Scalability, flexibility and performance
When you start using machine learning for real applications, it is recommended to consider the following questions before you start creating your solution architecture:
- Do you just want to experiment and play with some machine learning models?
- Do you want to try different machine learning frameworks and libraries in to discover what works best for your use case? Machine learning systems never work directly. You will need to iterate, rework and start all over again. Its innovation!
- Is performance crucial for your application?
- Are human lives direct or indirect dependent of your machine learning system?
Of course this reference architecture is an open architecture, so open for improvements and discussions.
Currently there is a awesome project that is all about collaborative learning on applying FOSS Machine Learning. This project is an open research project. It is open for all to join.