Founded in 1989, Millennium is a global alternative investment management firm. Millennium seeks to pursue a diverse array of investment strategies across industry sectors, asset classes and geographies. The firm’s primary investment areas are Fundamental Equity, Equity Arbitrage, Fixed Income, Commodities and Quantitative Strategies. We solve hard and interesting problems at the intersection of computer science, finance, and mathematics. We are focused on innovating and rapidly applying innovations to real world scenarios. This enables engineers to work on interesting problems, learn quickly and have deep impact to the firm and the business.
The Commodities Technology team builds and operates the data platform that aggregates and curates critical commodities data, including weather, supply/demand, storage, transportation and other fundamental and alternative datasets. This curated “content layer” is central to how our Portfolio Managers and researchers understand markets and construct trades.
The Role
We are seeking a Commodities Content Engineer who will focus on building robust ETL workflows and data models on top of our commodities data platform.
In this role, you will use Python and SQL to design, implement and maintain pipelines that ingest, clean, transform, and catalog commodities datasets. You will work closely with quantitative researchers, data analysts, and the broader Commodities Technology team to translate domain requirements into well‑structured, reliable data assets that can be easily discovered and reused across strategies.
This is a hands‑on engineering role with significant exposure to commodities data and the opportunity to shape how that data is represented and consumed across the firm.
Key Responsibilities
- ETL Development: Design and implement end‑to‑end ETL workflows in Python and SQL to ingest and transform commodities data from multiple vendors and internal sources.
- Data Modelling & Cataloging: Build and maintain standardized data models, schemas, and metadata that make commodities datasets easy to understand and discover within the platform.
- Workflow Orchestration: Use Airflow (or similar tools) to schedule, monitor, and manage data pipelines, ensuring reliability and timely delivery.
- Data Quality & Validation: Implement robust validation, reconciliation, and anomaly‑detection checks to ensure data completeness, correctness, and consistency.
- DevOps & Automation: Leverage Git, GitHub Actions, and automated testing (PyTest) to maintain high‑quality code and repeatable deployments.
- Collaboration with Domain Experts: Partner with commodities PMs, researchers, and data strategists to understand use cases and continuously refine datasets, definitions, and documentation.
Required Qualifications
- Experience: 2–5 years of experience in data engineering, analytics engineering, or similar roles focused on building and maintaining ETL pipelines.
- Programming & SQL: Strong skills in Python and SQL, with experience working with large datasets and complex transformations.
- Orchestration: Hands‑on experience with Airflow or other workflow schedulers.
- DevOps: Familiarity with version control (Git), CI/CD pipelines (GitHub Actions or equivalent), and test automation (e.g., PyTest).
- Data Mindset: Strong attention to detail, data quality and documentation; ability to reason about edge cases and data integrity.
- Soft Skills: Ability to work independently, communicate clearly with both technical and non‑technical stakeholders, and manage work across multiple concurrent initiatives.
Preferred Qualifications
- Knowledge of commodities markets and commodities data (e.g., weather, supply/demand, storage, freight, flows).
- Experience with data warehousing technologies (e.g., Snowflake, columnar storage formats, or analytic databases).
- Prior experience in a financial services, trading, or research‑driven environment.
- Exposure to data catalog / data governance tools and best practices.