Skip to content

📥 BRIM Architecture

BRIM (Broker Report Import Manager) is the system responsible for importing transaction data from CSV files exported by various brokers. It is designed to be robust, user-friendly, and extensible.

🔄 The BRIM Workflow

The import process follows a clear, multi-step workflow designed to give the user full control and visibility.

graph TD
    A[1. Upload] --> B{2. Storage & Analysis}
    B --> C[3. Parse & Preview]
    C --> D{4. User Review & Mapping}
    D --> E[5. Final Import]
    E --> F[(Database)]

    subgraph User Action
        A
        D
    end

  1. Upload: The user uploads a CSV file through the frontend.

  2. Storage & Analysis:

    • The backend saves the file with a unique UUID to prevent filename conflicts and path traversal attacks. The original filename is stored in a .json metadata sidecar file.
    • The file is placed in the backend/data/broker_reports/uploaded/ directory.
    • BRIM automatically analyzes the file to detect which plugins (BRIMProvider) can parse it.

  3. Parse & Preview:

    • The user selects a compatible plugin from the list.
    • The backend calls the plugin's parse() method, which reads the file and converts it into a standardized list of TXCreateItem objects.
    • Crucially, nothing is saved to the database at this stage.

  4. User Review & Mapping:

    • The frontend displays the parsed transactions to the user for review.
    • Asset Mapping: If the CSV contains new or unrecognized assets, the user is prompted to map them to existing assets in the database or create new ones.
    • Duplicate Detection: The system flags potential duplicate transactions, allowing the user to choose whether to skip or import them.

  5. Final Import:

    • Once the user confirms, the frontend sends the final, validated list of transactions to the backend.
    • The backend uses the standard TransactionService to save the transactions to the database.
    • The imported file is moved from the uploaded/ directory to parsed/ or failed/ depending on the outcome.

📂 File Lifecycle

BRIM manages imported files by moving them between three directories, ensuring a clear record of the import status.

  • backend/data/broker_reports/
    • uploaded/: New files awaiting processing.
    • parsed/: Files that have been successfully imported.
    • failed/: Files that failed to parse or were rejected by the user.

Each file is stored with a corresponding .json metadata file that tracks its status, original filename, and any errors.

🔍 Deduplication Logic

Before final import, BRIM checks for potential duplicates by comparing each parsed transaction against existing transactions in the database based on:

  • broker_id
  • date
  • type
  • quantity
  • amount

The system assigns a match confidence level to help the user make an informed decision:

  • POSSIBLE: Key fields match.
  • LIKELY: Key fields and the description match.
  • POSSIBLE_WITH_ASSET: Key fields match, and the asset was automatically resolved.
  • LIKELY_WITH_ASSET: Key fields, description, and the resolved asset all match.