Automating User Permissions Update in Databricks(Azure) Using Bash, jq, and curl

Managing user permissions in Databricks is a crucial aspect of ensuring secure and efficient data processing. In this blog post, we’ll explore how to automate the process of updating user permissions using a Bash script, jq for JSON parsing, and curl for making API requests.

Prerequisites

Before we begin, make sure you have the following:

Databricks instance URL (<databricks-instance>)
Databricks token (<databricks-token>)
Sample JSON file (input.json) containing user data

Bash Script Overview

The Bash script uses a for loop to iterate over each user object in the user array of the provided JSON file (input.json). It extracts the user_id and permissions using jq and then makes a curl request to update the user’s permissions in Databricks.

#!/bin/bash

# Set variables for Databricks instance and token
DB_INSTANCE="<databricks-instance>"
DB_TOKEN="<databricks-token>"

# Loop over each user and call curl
for user in $(jq -c '.user[]' input.json); do
  user_id=$(echo "$user" | jq -r '.user_id')
  permissions=$(echo "$user" | jq -r '.permissions[0] | @json')

  curl --netrc -X PATCH \
    "https://${DB_INSTANCE}/api/2.0/preview/scim/v2/Users/${user_id}" \
    --header 'Content-type: application/scim+json' \
    -H "Authorization: Bearer ${DB_TOKEN}" \
    --data "${permissions}"
done

Script Explanation

Variables Setup: Replace <databricks-instance> and <databricks-token> with your Databricks instance URL and token.
JSON Parsing with jq: The script uses jq to iterate over each user object, extracting user_id and permissions.
curl Request: The script then makes a curl request to the Databricks SCIM API endpoint, updating user permissions.

Example JSON File (input.json)

{
  "user": [
    {
      "user_id": "123",
      "permissions": [
        {
          "read": true,
          "write": false
        }
      ]
    },
    // Add more user objects as needed
  ]
}

Running the Script

Save the script to a file, e.g., update_permissions.sh.
Make it executable: chmod +x update_permissions.sh.
Run the script: ./update_permissions.sh.

This script provides a flexible way to automate user permissions updates in Databricks, making it easier to manage access control efficiently.

Happy scripting! 🚀

DevToolHub