BigQuery Jobs

Design & Architecture of a Secure Data Clean Room for Credit Bureau Data Exchange Project description We are building a Data Clean Room (DCR) to securely exchange and analyze third-party data with our credit bureau platform in Ecuador, ensuring data privacy, regulatory compliance, and zero raw data exposure. We are looking for a senior data engineer / data architect with hands-on experience designing Data Clean Rooms, preferably in financial services, credit bureaus, telecom, or regulated environments. This project is not just implementation — we need expert guidance to determine whether it is better to build the DCR in-house or use an existing commercial solution. What we need We want the expert to help us with: 1. Design a Data Clean Room architecture o Secure data ingestion fr...

Design & Architecture of a Secure Data Clean Room for Credit Bureau Data Exchange Project description We are building a Data Clean Room (DCR) to securely exchange and analyze third-party data with our credit bureau platform in Ecuador, ensuring data privacy, regulatory compliance, and zero raw data exposure. We are looking for a senior data engineer / data architect with hands-on experience designing Data Clean Rooms, preferably in financial services, credit bureaus, telecom, or regulated environments. This project is not just implementation — we need expert guidance to determine whether it is better to build the DCR in-house or use an existing commercial solution. What we need We want the expert to help us with: 1. Design a Data Clean Room architecture o Secure data ingestion fr...

I need a Google Cloud / BigQuery specialist to stand up an end-to-end, webhook-driven data ingestion pipeline running in our production environment. When our external form system fires a webhook, your Cloud Function (or equivalent service) should capture the JSON payload, write the untouched record to a raw BigQuery table, then immediately process it. The processing step must • parse any nested JSON, • flatten and clean each answer field, • split the results into two purpose-built reporting tables, and • guarantee idempotency through a hashing technique that blocks duplicates. All components have to be secure, version-controlled, and able to scale with traffic spikes. This is strictly a backend/data-engineering job—no UI work is involved.

My goal is to turn a messy collection of raw files into a clean, analysis-ready dataset hosted on Google Cloud so I can make faster, well-informed decisions. The data first needs three specific text-cleaning steps—removing duplicates, correcting typos, and standardising formats. Once those tasks are automated, the cleaned output should flow straight into a Google Cloud environment (BigQuery is ideal, but Cloud Storage plus Cloud Functions works too). After the upload, I want an exploratory analysis that highlights patterns, trends and any obvious outliers that deserve attention. Deliverables • Well-commented scripts or workflows that automate duplicate removal, spell-checking and format standardisation • A repeatable pipeline that loads the cleansed data into Google C...