2 min read

LetsDevOps: Create First Python Project, Build & Run locally.

Introduction

This series covers the fundamental idea of MLOps and demonstrates how to implement it with tools such as Azure DevOps and Azure Databricks.

It consists of three parts.

Part 1 involves demonstrating the MLOps concept, creating the sample, and installing and setting up locally.

Part 2 explains how to deploy and run the package manually on the Azure Databricks workspace.

Part 3 will concentrate on setting up the project on Git and creating the CI/CD to implement the MLOps flow.

What is MLOps

MLOps is a concept focused on effectively and reliably deploying and maintaining machine learning models in production.

To successfully implement MLOps, a solid understanding of both Machine Learning and DevOps is essential.

What is Machine Learning Models.

ML model is an object that has trained to recognize the pattern and later we use it to predict the output based on the input.

Workflow Architecture

Create Simple Python Project

In this project, we will develop a Python project that displays "Hello, World!" and proceed to set it up on our local environment. We will examine the project's structure, create the necessary artifact, install the package, and finally execute the program.

Prerequisites

Visual Studio Code (or any preferred IDE)
https://code.visualstudio.com/
Python installed locally
https://www.python.org/downloads/ During installation, make sure to check the box that says "Add Python to PATH".
Install Required Packages

pip install setuptools wheel

Create Project Folder Strucutre

ds_project/

│

├── ds_project/

│ ├── __init__.py

│ ├── module1.py

│

├── tests/

│ ├── __init__.py

│ ├── test_module1.py

│

├── setup.py

└── README.md

Add Main Function in module1.py

# ds_project/module1.py

def hello_world():

    return "Hello, world! My name is Sumit@@@"

def main():

    print(hello_world())

if __name__ == "__main__":

    main()

Create unit test test_module1.py

import unittest

from ds_project.module1 import hello_world

class TestModule1(unittest.TestCase):

    def test_hello_world(self):

        expected_output = "Hello, world! My name is Sumit@@@"

        self.assertEqual(hello_world(), expected_output)

if __name__ == '__main__':

    unittest.main()

Create setup.py

from setuptools import setup, find_packages

setup(

    name="ds_project",

    version="0.1",

    packages=find_packages(),

    install_requires=[

        # List your project dependencies here

],

    entry_points={

        'console_scripts': [

            'hello-world=ds_project.module1:main',

],

},

Build the Package

python setup.py sdist bdist_wheel

Install the package locally

pip install dist/ds_project-0.1-py3-none-any.whl

Run the Entry Point

hello-world

Running the unit test

python -m unittest discover -s tests

Additional Command for Debugging

--> remove the build

rm -rf build dist ds_project.egg-info

--> Verify the Installation

pip show ds_project

--> Uninstall

pip uninstall ds_project

Demo Project : https://github.com/sumitraj0103/Letsdevops/tree/main/mlops_demo/ds_project

Troubleshooting

Error 1: pip is not recognised

Resolution 1: add the installed python pip path to to environment variable and restart the machine.

Path: C:\Users\srs\AppData\Local\Programs\Python\Python312\Scripts

srs is my user you can get the path for your user and update.

Add the path in the Environment Variable and restart the machine

Error 2: python is not recognised

Resolution 2: add the installed python pip path to to environment variable and restart the machine.

Path: C:\Users\srs\AppData\Local\Programs\Python\Python312