GCP with Datastream_ Chennai, Hyderabad, Bangalore, Pune,Kolkata

Full Time 1 week ago
Employment Information

Experience in GCP data development.

  • Hands-on experience with Data-stream, Data-flow, Pub/Sub, Big-Query, and Cloud Storage.
  • Strong knowledge of SQL and Python;
  • familiarity with Java/Scala is a plus.
  • Experience with Terraform for Infrastructure as Code (IAC)
  • Excellent communication and analytical skills.
  • Ability to work under pressure and manage deadlines.

Data Streaming & Integration

  • Develop and maintain Data-stream pipelines for change data capture (CDC) and replication from sources like Oracle, MySQL, PostgreSQL, and SQL Server into Big-Query, Cloud Storage, and other destinations.
  • Integrate Data-stream with Dataflow templates for custom transformations and event-driven architectures.

Pipeline Development & Orchestration

  • Build ELT workflows using Data-stream, Pub/Sub, Data Fusion, and Cloud Composer.
  • Implement monitoring and alerting for streaming pipelines to ensure reliability and minimal latency.

Performance & Optimization

  • Optimize streaming jobs for cost and performance efficiency.
  • Ensure high availability and fault tolerance in data replication processes.

Security & Compliance

  • Implement IAM policies, private connectivity options, and encryption for secure data transfer.
  • Maintain documentation for architecture, configurations, and operational procedures.