Abstract

Machine Learning (ML) has gained prominence and has tremendous applications in fields like medicine, biology, geography and astrophysics, to name a few. Arguably, in such areas, it is used by domain experts, who are not necessarily skilled-programmers. Thus, it presents a steep learning curve for such domain experts in programming ML applications. To overcome this and foster widespread adoption of ML techniques, we propose to equip them with domain-specific graphical tools. Such tools, based on the principles of flow-based programming paradigm, would support the graphical composition of ML applications at a higher level of abstraction and auto-generation of target code. Accordingly, (i) we have modelled ML algorithms as composable components; (ii) described an approach to parse a flow created by connecting several such composable components and use an API-based code generation technique to generate the ML application. To demonstrate the feasibility of our conceptual approach, we have modelled the APIs of Apache Spark ML as composable components and validated it in three use-cases. The use-cases are designed to capture the ease of program specification at a higher abstraction level, easy parametrisation of ML APIs, auto-generation of the ML application and auto-validation of the generated model for better prediction accuracy.

Highlights

  • Machine Learning (ML) is a scientific discipline that develops and makes use of a particular class of algorithms which are designed to solve problems without explicit programming [1]

  • ML techniques used by domain experts like traffic engineers or molecular biologists who are less-skilled programmers have to counter a steep learning curve, i.e., learn how to program and write an ML application from scratch using general-purpose, high-level languages like Java, Scala or Python

  • The use-cases demonstrate how such flows can be created by connecting different components from step 1 at a higher level of abstraction, parameters to various components can be configured with ease, automatic parsing of the user flow to give feedback to the user if a component has been used in a wrong position in a flow and automatic generation of ML application without the end-user having to write any code

Read more

Summary

Introduction

Machine Learning (ML) is a scientific discipline that develops and makes use of a particular class of algorithms which are designed to solve problems without explicit programming [1]. The algorithms infer about patterns present in a dataset and learn how to solve a specific problem This self-learning technique of computer systems has gained prominence and has vast application in the current era. We intend to support graphical specification of ML programs via a flow-based programming paradigm and support auto-generation of target code, thereby shielding the user of such tools from nuances and complexities of programming. Such graphical flow-based programming tools called mashup tools have been extensively used to simplify application development [2].

Contributions
Outline
Machine Learning
Machine Learning Libraries
Flow-Based Programming
Model-Driven Software Development
Related Work
Apache Spark
ML with Spark
Design Choice
Conceptual Approach
Flow Specification and Flow-Checking
Model Generation
Model Evaluation and Hyperparameter Tuning
Prototype Overview
Components
Working
Running Examples
Use Case 1
Use Case 2
Discussion
Comments about Previous Attempts
Findings
Advantages and Limitations
Conclusions
Full Text
Paper version not known

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.