LetsDevOps: Azure Databricks Git Integration, Create Branch, Pull Request and Merge the Changes.

Introduction


In this blog we will learn How to integrate Azure Databricks with Git. Further we will learn How to Create Branch, Merge the change and Create Pull Request using the Azure Databricks Console.


There are multiple Git Provider that can be used for the Git Integration

  • GitHub

  • GitLab

  • Azure DevOps

  • Jenkins



Azure Databricks Git Integration with GithHub

In this section we will learn to setup Azure Databricks Git Integration with GitHub.


How to Setup

Step 1:


Make sure to have below information.


1. Git User Setting [Git provider username or email]

2. PAT [Personal Access Token]

GitHub User Setting --> Developer Setting --> Personal Access Token




Step 2

Connect to Azure Databricks workspace --> Setting --> User Setting --> Git Integration and Save the setting. You can use any one of the Git Provider. In this example we have used GitHub.




Step 3

Once you have saved the user setting, Now its time to Clone the GitHub Repo in Azure Databricks Console.


Databricks --> Repos --> Add Repo --> Clone Repo


Note: Make sure the user setting Account has access to the repo.




Step 4:

After the successful Creation, you can see the repo gets added.




Create Feature Branch Merge the Change and Create Pull Request.


Once the Git Integration completes now we can learn how to create Feature Branch, complete the Changes and Create Pull Request to complete the Merge.


Create Feature Branch


1. Click on the Drop Down at the Branch Name and Select Git





2. Now Create the Demo-feature Branch to start working on the changes. Make sure to pull the Latest code before create the feature branch.




3. Next, I will create one sample Python File and commit & Push the changes to Feature Branch.





4. After the successful Commit and Push, Once you login to GitHub you can see the Branch and change request has been pushed




5. You can create Pull Request and Complete the Merge.




6. After the Merge you can see the Demo Files merged to Main Branch.




GitHub Demo Project:

https://github.com/sumitraj0103/Azure-Databricks-CICD