Available for new work Data Engineer · Sydney, Australia

Data pipelines that hold up in production.

I'm Richard Friedman — a data engineer with 7+ years designing scalable ETL/ELT pipelines and cloud-native platforms across enterprise environments. Python, PySpark and SQL at the core; certified across Databricks, AWS, Snowflake and Airflow.

7+
Years in data engineering
9
Industry certifications
5
Cloud & data platforms

About

I turn business requirements into robust, maintainable data architectures — and reliable pipelines the team can run without me.

Across enterprise and consulting work I've built ETL/ELT pipelines, cloud-native platforms and ML systems that cut processing times, automate manual workflows and raise data quality for stakeholders. My foundation is CI/CD, Docker and infrastructure-as-code — the unglamorous parts that keep a system dependable in production.

Programming

  • Python
  • SQL
  • JavaScript
  • C#
  • C

Data engineering

  • PySpark
  • PostgreSQL
  • ETL / ELT
  • Databricks
  • Snowflake
  • Airflow
  • Kestra

Machine learning

  • scikit-learn
  • TensorFlow
  • Keras
  • PyTorch
  • Time-series

Cloud & infra

  • AWS
  • Azure
  • GCP
  • Docker
  • Terraform
  • CI/CD
  • Git

Analytics & BI

  • Pandas
  • NumPy
  • Streamlit
  • Flask
  • Power BI
  • Tableau

Practices

  • Agile
  • Data quality
  • Data governance
  • Documentation

Certifications

  • AWS Cloud Practitioner
  • AWS AI Practitioner
  • Azure Fundamentals
  • Databricks Data Engineering
  • Databricks Fundamentals
  • Databricks GenAI
  • Databricks AI Agent
  • Airflow 3 Fundamentals
  • Snowflake — Zero to Snowflake

Experience

Where I've shipped, and what came of it.

  • Imagineered Consulting

    Principal Data Consultant & Sports Trader

    2015 – 2025

    Built and ran end-to-end analytics pipelines for sports-betting markets on Databricks — API ingestion, PySpark ETL, predictive ML models and automated alerting delivered to subscription clients. Engineered deep-learning architectures to process high-frequency odds data for near real-time feature engineering and backtesting across millions of events.

    → Sustained a profitable edge against efficient global markets over a 10-year period, measured by positive closing line value (CLV).

    • Databricks
    • PySpark
    • Python
    • ML
    • Time-series
  • Ventia Australia

    Data Engineer

    Nov 2021 – 2023

    Designed ETL pipelines for NBN data validation and GIS extraction, and built cloud-based Python applications that turned manual workflows into self-service tools and automated reporting dashboards.

    → Cut design errors 30% and processing time from 8h to 1h. Eliminated 40% of repetitive tasks and reduced downstream rework 25%.

    • Python
    • PostgreSQL
    • ETL
    • Azure
    • Power BI
  • Insight Analytics

    Data Analyst

    2020 – 2021

    Queried and validated data from central databases with SQL to support client deliverables under tight deadlines, operating fully remote with secure data transfers and compliance for sensitive datasets.

    → Streamlined high-volume data workflows on Databricks for faster stakeholder decision-making.

    • SQL
    • Databricks
    • Cloud
  • SIRCA Australia

    Data Scientist

    2019

    Orchestrated Hadoop workloads to ingest and transform terabytes of structured and unstructured financial data, with secure server-side operations over SSH and full audit trails.

    → Delivered client data requests from centralised systems to SLA, with accuracy and completeness verified end to end.

    • Hadoop
    • Python
    • SQL
    • SSH
  • UTS Summer Studio

    Data Scientist & Project Leader

    2012 – 2013

    Led a 4-person agile team to build an LSTM Forex-forecasting prototype with TensorFlow/Keras, deployed in a Flask web application for live stakeholder demonstrations.

    → Presented results at an industry showcase event.

    • TensorFlow
    • Keras
    • Flask
    • LSTM

Contact

Let's talk.

Tell me about the data problem you're working on and what good looks like. I read every message and reply within a working day.

Based
Sydney, Australia · AEST
Status
Australian citizen
Open to
Data engineering roles & contracts