Experience in GCP data development.
- Hands-on experience with Data-stream, Data-flow, Pub/Sub, Big-Query, and Cloud Storage.
- Strong knowledge of SQL and Python;
- familiarity with Java/Scala is a plus.
- Experience with Terraform for Infrastructure as Code (IAC)
- Excellent communication and analytical skills.
- Ability to work under pressure and manage deadlines.
Data Streaming & Integration
- Develop and maintain Data-stream pipelines for change data capture (CDC) and replication from sources like Oracle, MySQL, PostgreSQL, and SQL Server into Big-Query, Cloud Storage, and other destinations.
- Integrate Data-stream with Dataflow templates for custom transformations and event-driven architectures.
Pipeline Development & Orchestration
- Build ELT workflows using Data-stream, Pub/Sub, Data Fusion, and Cloud Composer.
- Implement monitoring and alerting for streaming pipelines to ensure reliability and minimal latency.
Performance & Optimization
- Optimize streaming jobs for cost and performance efficiency.
- Ensure high availability and fault tolerance in data replication processes.
Security & Compliance
- Implement IAM policies, private connectivity options, and encryption for secure data transfer.
- Maintain documentation for architecture, configurations, and operational procedures.