Building gradle dependency graph using Python and GraphViz

March 03, 2023

Problem

In this article we’ll demonstrate how to build graph visualization for a gradle project structure, to see which modules depend on other modules.

Some Gradle projects can become big and it becomes hard to make sense of the structure, by visualizing it you can easily see which modules are unnecessary.

Let’s say we have a project with 3 subprojects inside api, shared and services (from https://docs.gradle.org/current/userguide/declaring_dependencies_between_subprojects.html)

api/build.gradle

dependencies {
    implementation project(':shared')
}

shared/build.gradle

dependencies {
  implementation "some:opensource:2.13"
}

services/build.gradle

dependencies {
    implementation project(':shared')
    implementation project(':api')
}

Implementation

Here is a task break down

Python Command line tool that takes project location as input
Find all Gradle files and their module names
- We'll scan a directory for all build.gradle files
- Project name will always be a parent folder name
We’ll parse each gradle file looking for some keywords like "project" and
Construct a structure {name -> [dependencies]}
Convert that structure into DOT markup
Preview result

Create python command line tool

this is fairly standard:

if __name__ == "__main__":
  project_path = sys.argv[1]
  print("checking project: ", project_path)

output

python3 generate_graph.py /Users/devtoolsdaily/src/gradleproj
checking project:  /Users/devtoolsdaily/src/gradleproj

Find all Gradle files and their module names

for that we'll use pathlib.Path.rglob function that can find all files by name, then we'll parse their parent folder using os.path.split

def find_modules(project_path):
  project_to_gradle_file = {}
  for full_path in pathlib.Path(project_path).rglob("build.gradle"):
      folder = os.path.split(full_path)[0]
      if folder == project_path:
          # skipping root build.gradle
          continue
      module_name = os.path.split(folder)[1]
      # initializing our result with empty dependencies first
      module_to_gradle_file[module_name] = full_path
  return module_to_gradle_file

if __name__ == "__main__":
  project_path = sys.argv[1]
  print("checking project: ", project_path)

  modules_to_gradle_files = find_modules(project_path)
  print(modules_to_gradle_files)

output

{'shared': PosixPath('/Users/devtoolsdaily/src/gradleproj/shared/build.gradle'),
'api': PosixPath('/Users/devtoolsdaily/src/gradleproj/api/build.gradle'),
'services': PosixPath('/Users/devtoolsdaily/src/gradleproj/services/build.gradle')}

We’ll parse each gradle file looking for some keywords like "project" and

for each line in gradle file, we'll check for other project names and for keyword project

def find_internal_dependencies(gradle_file_path, all_modules):
  dependencies = []
  with open(gradle_file_path) as f:
      for line in f:
          print(line)
          for module in all_modules:
              if module in line and "project" in line:
                  dependencies.append(module)
  return dependencies

We'll construct a final structure out of that

if __name__ == "__main__":
  project_path = sys.argv[1]
  print("checking project: ", project_path)
  modules_to_gradle_files = find_modules(project_path)
  
  module_dependencies = {}
  for module_name, gradle_file in modules_to_gradle_files.items():
      dependencies = find_internal_dependencies(gradle_file, modules_to_gradle_files.keys())
      module_dependencies[module_name] = dependencies
  print(module_dependencies)

that outputs

{'shared': [], 'api': ['shared'], 'services': ['api', 'shared']}

Convert that structure into DOT markup

def convert_to_dot(module_dependencies):
  result = ["digraph {"]
  for module, dependencies in module_dependencies.items():
      for dependency in dependencies:
          result.append(f"  {module} -> {dependency}")
  result.append("}")
  return "\n".join(result)

digraph {
  api -> shared
  services -> api
  services -> shared
}