Senior Data Engineer / Madrid

Joshua de Freitas

I design data platforms and the systems that keep them dependable — pipeline architecture, analytical workflows and applied AI for organisations with real complexity to handle.

The public work here spans deterministic simulation, neural architectures, WebGPU, market microstructure and developer tooling. Each project carries its evidence and its limits.

Gray–Scott 0 steps · deterministic replay

01 — Work

Selected projects

Small, complete systems, published with enough evidence for someone else to test them, extend them, or find the mistake.

Studies

Small experiments built to answer one question at a time. The result matters, but so does showing where it stops.

Neural Field Study

Computational neuroscience

Imagine a sheet of cells. Each active cell encourages its nearest neighbours and suppresses a wider ring around it. This study lets that simple rule run, then checks which patterns come from the rule and which depend on numerical choices such as the grid’s edges.

Does not establish how real brains behave. It is a controlled synthetic system for studying how local rules create global patterns.

JavaScriptDeterministic simulationNumerical audit

DeepLOB

Evaluation integrity

Many market models train on overlapping slices of the same timeline. If near-duplicates appear in both training and validation, the model can seem predictive because it has effectively seen the answer already. This study measures that leak and removes it.

Does not establish tradability, profit or a real market signal. It shows how a flawed test can manufacture confidence.

PythonPyTorchNegative-control study

Tools

Less visible, more used. Built because I needed them and nothing else would do the job.

trace-npm catching a package that reads ~/.ssh/id_rsa and sends it to a remote host

trace-npm

Supply chain forensicsAlpha on npm

npm now asks you to approve install scripts. Nothing tells you what they do. trace-npm runs one under a syscall tracer and reports what it read, spawned and contacted — including the planted credential it took the bait on.

npx trace-npm run --package suspect --script postinstall

Does not establish that a package is safe. Evidence, not verdicts — and the report lists what it could not see.

Node.jsLinux · straceNo dependencies

dbx-core

Data engineering

Databricks without the notebook sprawl: real Python packages, local tests, remote execution. The foundation a data platform should actually rest on.

pytest tests/ && dbx deploy

Opinionated about jobs over notebooks. That constraint is the point, and it will not suit every team.

PythonDatabricksJobs over notebooks

02 — Practice

How I work

I like hard problems, but I do not think the explanation has to be hard. I try to understand the decision first, build the smallest useful system, and leave it in a state someone else can actually own.

  1. 01
    Understand the real problem

    A polished answer to the wrong question is still wrong. I spend time with the people, data and constraints before reaching for an architecture.

  2. 02
    Make the work easy to inspect

    Clear assumptions, reproducible runs and tests that fail loudly. Clever is useful; legible is better.

  3. 03
    Leave it easier to own

    The job is not finished when the demo works. It is finished when the system can be operated, questioned and changed without its author in the room.

03 — About

Background

I am a senior data engineer and AI practitioner based in Madrid, working at EY. I build the data foundations, analytical systems and applied AI workflows that translate complex operational and economic problems into measurable signals, testable models and reliable decisions. The work sits where data architecture, software engineering and quantitative reasoning meet — and it has to remain dependable under real production constraints.

I am especially drawn to business and economic problems that become tractable through computation: how information moves through an organisation, how incentives and uncertainty shape outcomes, and how models can support decisions without hiding their assumptions. That interest carries into the public work here — deterministic simulation, neural architectures, market microstructure, WebGPU and developer tooling — each project built to make its mechanism, evidence and limitations inspectable.

These projects are personal, publicly released work and separate from my role at EY. This site is not a solicitation for employment or consulting.

Currently
Building a WebGPU compute engine, writing up the neural field study, and maintaining trace-npm on npm. August 2026
Languages & data
Python, SQL, PySpark, TypeScript
Platform
Databricks, Delta Lake, Neo4j, APIs
Research
Machine learning, simulation, WebGPU, visualisation

04 — Contact

Contact

Based in Madrid. Questions about the published work are welcome.

jamesdefreitas@outlook.com