TASK3_MLOPS_MACHINE_LEARNING_AND_TWEAKING_USING_AUTOMATION
TASK DESCRIPTION:-
Create container image that’s has Python3 and Keras or numpy installed using dockerfile.
When we launch this image, it should automatically starts train the model in the container.
Job1 : Pull the Github repo automatically when some developers push repo to Github
Job2 : By looking at the code or program file, Jenkins should automatically start the respective machine learning software installed interpreter install image container to deploy code and start training( eg. If code uses CNN, then Jenkins should start the container that has already installed all the softwares required for the cnn processing).
In my code the jenkins job will search for the keyword keras in the python code file for the cnn model and the sklearn keyword for the rnn model because these are the main libraries for the model and then based on that search it will go inside the if-else block and run diff containers. I have created two different containers, the first in which i have installed all the module like tensorflow , keras ,numpy ,pandas ,opencv-python etc and in the same way i have created a different container to train the regression model.
Job3 : Train your model and predict accuracy or metrics.
Job4 : if metrics accuracy is less than 80% , then tweak the machine learning model architecture.
In this part I have merged these two jobs using the python code but in my previous posts on linkedin you can find a complete docx file in which i have explained these two jobs running in the same order.
In the python code I have just created a text file and stored the value of accuracy in it. And based on that value it will further train the model.
However i have retrained my model in my windows directly from the python code and the accuracy incereased from 42% to 84% in just 3 loops of epochs.
Note- in this code i have only changed the hyperparameters for the epochs because if i change the no of neurons it will create a whole new FC layer without removing the last output layer. I have tried to create a new code for that part and i will soon upload that part on my github account.
Job5: Notify that the best model is being created
Create One extra job job6 for monitor : If container where app is running. fails due to any reason then this job should automatically start the container again from where the last trained model left
I have completed this project with my friend Aditya Sharma
For referece you can check the github account
LinkedIN profile: https://www.linkedin.com/in/akshat-soni-011b461a6