Development Workflows

Overview

This guide covers common workflows for developing with IVPM, from project initialization through daily development and release preparation.

Project Lifecycle

Starting a New Project

Step 1: Initialize

$ mkdir my-project
$ cd my-project
$ git init
$ ivpm init my-project -v 0.1.0

Step 2: Configure Dependencies

Edit ivpm.yaml:

package:
  name: my-project
  version: "0.1.0"
  default-dep-set: default-dev

  dep-sets:
    - name: default
      deps:
        - name: requests
          src: pypi

    - name: default-dev
      deps:
        - name: requests
          src: pypi
        - name: pytest
          src: pypi
        - name: black
          src: pypi

Step 3: Create Project Structure

$ mkdir -p src/my_project test docs
$ touch src/my_project/__init__.py
$ touch test/test_basic.py
$ echo "# My Project" > README.md
$ echo "packages/" > .gitignore
$ echo "build/" >> .gitignore
$ echo "__pycache__/" >> .gitignore

Step 4: Initialize Dependencies

$ ivpm update

Step 5: Initial Commit

$ git add ivpm.yaml src/ test/ README.md .gitignore
$ git commit -m "Initial project setup"

Cloning an Existing Project

For end users:

$ ivpm clone https://github.com/org/project.git
$ cd project
# Ready to use!

For developers:

$ ivpm clone https://github.com/org/project.git -b develop
$ cd project
$ ivpm activate
(venv) $ pytest

Working with Dependencies

Adding a New Dependency

Step 1: Add to ivpm.yaml

deps:
  - name: numpy
    src: pypi
    version: ">=1.20"

Step 2: Update

$ ivpm update

Step 3: Verify

$ ivpm activate -c "python -c 'import numpy; print(numpy.__version__)'"

Adding a Git Dependency

deps:
  - name: my-library
    url: https://github.com/org/my-library.git
    branch: main

$ ivpm update
$ ls packages/my-library  # Verify it's there

Adding a Local Development Dependency

For co-development of multiple projects:

deps:
  - name: shared-lib
    url: file:///home/user/projects/shared-lib
    src: dir
    link: true

$ ivpm update
$ ls -l packages/shared-lib  # Should be a symlink

Removing a Dependency

Step 1: Remove from ivpm.yaml

Delete the dependency entry.

Step 2: Clean up

$ rm -rf packages/dependency-name
$ ivpm update --force-py-install  # If it was a Python package

Updating Dependency Versions

For PyPI packages:

# Before
- name: requests
  src: pypi
  version: ">=2.28"

# After
- name: requests
  src: pypi
  version: ">=2.31"

$ ivpm update --force-py-install

For Git packages:

# Before
- name: lib
  url: https://github.com/org/lib.git
  tag: v1.0.0

# After
- name: lib
  url: https://github.com/org/lib.git
  tag: v1.1.0

$ rm -rf packages/lib
$ ivpm update

Daily Development

Morning Routine

$ cd my-project
$ git pull
$ ivpm update  # Get any new dependencies
$ ivpm status  # Check Git package status
$ ivpm activate

Interactive Development

# Start environment
$ ivpm activate
(venv) $

# Run your code
(venv) $ python src/main.py

# Run tests
(venv) $ pytest

# Format code
(venv) $ black src/

# Type check
(venv) $ mypy src/

# Done for the session
(venv) $ exit

Running Quick Commands

# No need to stay in activated environment
$ ivpm activate -c "pytest"
$ ivpm activate -c "python script.py"
$ ivpm activate -c "black --check src/"

Working with Editable Packages

Modifying Dependency Source

When a dependency is installed in editable mode (Git source):

$ cd packages/my-library
$ git status
$ # Make changes
$ git add .
$ git commit -m "Fix bug"
$ git push

Changes are immediately available to your project.

Testing Changes Before Committing

$ cd packages/my-library
$ # Edit files
$ cd ../..
$ ivpm activate -c "pytest"  # Uses modified code

Creating a Branch in a Dependency

$ cd packages/my-library
$ git checkout -b feature/new-thing
$ # Make changes
$ git push -u origin feature/new-thing
$ cd ../..

Update ivpm.yaml to use the new branch:

- name: my-library
  url: https://github.com/org/my-library.git
  branch: feature/new-thing

Submitting Upstream Changes

# 1. Make changes in packages/my-library
$ cd packages/my-library
$ git checkout -b fix/issue-123
$ # Make changes
$ git commit -m "Fix issue 123"
$ git push -u origin fix/issue-123

# 2. Create pull request on GitHub

# 3. After merge, update your project
$ cd ../..
$ vim ivpm.yaml  # Remove branch override if used
$ ivpm sync  # Or rm -rf packages/my-library && ivpm update

Git Package Management

Checking Package Status

$ ivpm status

Output shows:

Modified files
Uncommitted changes
Branch information
Ahead/behind upstream

Example output:

Package: my-library
  Branch: main
  Status: Clean

Package: test-utils
  Branch: develop
  Status: Modified
  M  src/utils.py
  ?? new_file.txt

Syncing with Upstream

Update all Git packages from their upstream:

$ ivpm sync

This runs git fetch and git merge for each Git package.

Selective sync:

$ cd packages/my-library
$ git pull
$ cd ../..

Handling Merge Conflicts

If ivpm sync encounters conflicts:

$ ivpm sync
# Error: Merge conflict in packages/my-library

$ cd packages/my-library
$ git status
$ # Resolve conflicts
$ git add .
$ git commit
$ cd ../..

Switching Branches

Temporary branch switch:

$ cd packages/my-library
$ git checkout feature-branch
$ cd ../..
$ ivpm activate -c "pytest"

Permanent branch switch:

Update ivpm.yaml:

- name: my-library
  url: https://github.com/org/my-library.git
  branch: feature-branch

$ rm -rf packages/my-library
$ ivpm update

Building and Testing

Building Python Extensions

For packages with native extensions:

$ ivpm build

Debug build:

$ ivpm build --debug

Specific dependency set:

$ ivpm build -d default-dev

Running Tests

# All tests
$ ivpm activate -c "pytest"

# Specific test file
$ ivpm activate -c "pytest test/test_feature.py"

# With coverage
$ ivpm activate -c "pytest --cov=src"

# Verbose
$ ivpm activate -c "pytest -v"

Running Linters and Formatters

# Format code
$ ivpm activate -c "black src/ test/"

# Check formatting
$ ivpm activate -c "black --check src/"

# Type checking
$ ivpm activate -c "mypy src/"

# Linting
$ ivpm activate -c "pylint src/"
$ ivpm activate -c "flake8 src/"

Continuous Integration

GitHub Actions example:

name: CI

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'

      - name: Install IVPM
        run: pip install ivpm

      - name: Update dependencies
        run: ivpm update -d default-dev

      - name: Run tests
        run: ivpm activate -c "pytest --cov=src"

      - name: Check formatting
        run: ivpm activate -c "black --check src/"

Release Workflows

Preparing a Release

Step 1: Switch to release dependencies

$ ivpm update -d default

Step 2: Run full test suite

$ ivpm activate -c "pytest"

Step 3: Build documentation

$ ivpm activate -c "sphinx-build docs/source docs/build"

Step 4: Update version

package:
  name: my-project
  version: "1.0.0"  # Update version

Step 5: Commit and tag

$ git add ivpm.yaml
$ git commit -m "Release v1.0.0"
$ git tag -a v1.0.0 -m "Release version 1.0.0"
$ git push origin main --tags

Creating a Snapshot

Create a self-contained copy of your project with all dependencies:

$ ivpm snapshot /path/to/snapshot-dir

This creates a directory with:

Project source
All dependencies
Python packages list (python_pkgs.txt)
Updated ivpm.yaml with exact versions

Use case: Archival, reproducible builds, offline development

Deploying to Users

Option 1: Users clone with IVPM

$ ivpm clone https://github.com/org/project.git

Option 2: Traditional clone + update

$ git clone https://github.com/org/project.git
$ cd project
$ pip install ivpm
$ ivpm update

Option 3: Snapshot distribution

$ ivpm snapshot release-v1.0
$ tar czf project-v1.0.tar.gz release-v1.0/
# Distribute tarball

Team Workflows

Onboarding New Team Members

# New member's machine
$ pip install ivpm
$ ivpm clone https://github.com/company/project.git
$ cd project
$ ivpm activate
(venv) $ pytest
# Ready to develop!

Shared Cache Setup

# Admin sets up shared cache
$ sudo mkdir -p /shared/ivpm-cache
$ sudo ivpm cache init --shared /shared/ivpm-cache
$ sudo chown :devteam /shared/ivpm-cache

# Team members add to ~/.bashrc
export IVPM_CACHE=/shared/ivpm-cache

Benefits:

First person downloads, everyone else gets instant symlinks
Saves bandwidth and disk space
Faster project initialization

Code Review Workflow

Reviewer:

$ git checkout review-branch
$ ivpm update  # Get any new dependencies
$ ivpm activate -c "pytest"  # Verify tests pass
$ ivpm activate -c "black --check src/"  # Check formatting

Feature Branch Workflow

# Start feature
$ git checkout -b feature/awesome
$ vim ivpm.yaml  # Add any needed dependencies
$ ivpm update

# Develop
$ ivpm activate
(venv) $ # work work work
(venv) $ pytest
(venv) $ exit

# Commit
$ git add .
$ git commit -m "Add awesome feature"
$ git push -u origin feature/awesome

# After merge, cleanup
$ git checkout main
$ git pull
$ ivpm update

Common Scenarios

Scenario 1: Dependency Has a Bug

Quick fix:

$ cd packages/buggy-lib
$ git checkout -b fix/bug-123
$ # Fix the bug
$ git commit -m "Fix bug"
$ cd ../..
$ ivpm activate -c "pytest"  # Test with fix

Update project to use fix:

- name: buggy-lib
  url: https://github.com/org/buggy-lib.git
  branch: fix/bug-123

Scenario 2: Need Older Version Temporarily

$ cd packages/my-lib
$ git checkout v1.0.0
$ cd ../..
$ ivpm activate -c "pytest"  # Test with old version

# Restore
$ cd packages/my-lib
$ git checkout main
$ cd ../..

Scenario 3: Working on Multiple Projects

# Project A uses library in development mode
$ cd project-a
$ vim ivpm.yaml
# - name: shared-lib
#   url: file:///home/user/projects/shared-lib
#   src: dir
$ ivpm update

# Edit library
$ cd ~/projects/shared-lib
$ # Make changes

# Test in Project A
$ cd ~/projects/project-a
$ ivpm activate -c "pytest"  # Uses modified library

# Test in Project B
$ cd ~/projects/project-b
$ ivpm activate -c "pytest"  # Also uses modified library

Scenario 4: Cleaning Up Stale Dependencies

# Remove packages directory
$ rm -rf packages/

# Recreate from scratch
$ ivpm update

Best Practices

Commit ivpm.yaml - Always version control your dependency configuration
Use dependency sets - Separate dev and release dependencies
Pin versions for releases - Use exact versions or tags for reproducibility
Use ranges for development - Allow flexibility with >= version specs
Document custom workflows - Add team-specific instructions to README
Regular updates - Run ivpm status and ivpm sync periodically
Cache wisely - Use cache: true for stable dependencies
Test before committing - Always run tests after dependency changes
Clean builds periodically - rm -rf packages/ && ivpm update
Use .gitignore - Never commit packages/ directory

Troubleshooting Common Issues

“Package Already Loaded” But Not Visible

$ rm -rf packages/problem-package
$ ivpm update

Dependency Version Conflicts

Check what’s installed:

$ ivpm activate -c "pip list"

Adjust version requirements in ivpm.yaml.

Git Package Won’t Sync

$ cd packages/problem-package
$ git status  # Check for uncommitted changes
$ git stash   # Save local changes
$ cd ../..
$ ivpm sync

Python Package Not Importable

$ ivpm update --force-py-install
$ ivpm activate -c "pip list"  # Verify it's installed