We are looking for a Sr Data Engineer who is passionate about managing large amounts of data, eager to learn and accept challenges, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then we want to hear from you
Roles and Responsibilities:
· Experience working with AWS service for data ETL
· 3-5 yrs years of Experience in developing data ingestion, data processing, and analytical pipelines for big data, relational databases, NoSQL, and data warehouse solutions
· Experience in designing data models effectively to manage optimal performance.
· Designing, developing, monitoring, and maintaining end-to-end data pipelines
· Should have an extensive working knowledge of handling around 100GB of data in data ingestion, data processing and pipeline creation.
· Experience with designing and implementing complex high throughput distributed batch and/or real-time solutions using one of AWS Glue, Apache Kafka, AWS Kinesis, Spark, Flink and Apache Airflow
· Extensive hands-on experience implementing data migration, data processing, ETL, or ELT pipeline development and monitoring.
· Strong programming skills in Python, SQL skills for data analysis and reporting
· Should know about any Data Warehouse like Redshift, Snowflake, and BigQuery
· Proven ability to work independently, mentor peers and meet tight deadlines
· Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases like Postgres, SQL Server, MySQL
· Should know about NoSQL Database like MongoDB, Cassandra, Neptune
· Experience in writing complex queries, functions, store procedures, execution plans, Performance Tuning and Query Optimization, Indexing, Partitioning, and Denormalization
· Should be well versed with migration strategies by manual and automated processes
· Excellent interpersonal, communication skills (both verbal and written), and ability to interact effectively
· Should attend client calls independently
· AWS certification would be a big plus