Sr. Data Engineer

Company: Softpath Technologies LLC
Location: Oakland
Posted on: November 20, 2020

Job Description:

We currently have the below urgent requirement pls reach me at kpadmajaATsoftpathtech.com with suitable profiles RoleSr. Data Engineer LocationNewark , CA Exp6Yrs Data Engg and BI IOT Tools - MQTT, Kafka, Apache Spark and data frame Languages Java, Python 6 years of experience in Data Engineering and Business Intelligence. Proficient in IoT tools such as MQTT, Kafka, Spark Proficient with AWS, S3, Redshift Experience with Presto and ParquetORC Proficient with Apache Spark and data frame. Experienced in containerization, including Docker and Kubernetes Expert in tools such as Apache Spark, Apache Airflow, Presto Expert in design and implement reliable, scalable, and performant distributed systems and data pipelines Extensive programming and software engineering experience, especially in Java, Python, Experience with Columnar database such as Redshift, Vertica Hands-on design and develop streaming and IoT data pipelines. Developing streaming pipeline using MQTT, Kafka, Spark Structure Streaming Orchestrate and monitor pipelines using Prometheus and Kubernetes Deploy and maintain streaming jobs in CICD and relevant tools. Python scripting for automation and application development Design and implement Apache Airflow and other dependency enforcement and scheduling tools. Hands-on data modeling and data warehousing Deploy solution using AWS, S3, Redshift and DockerKubernetes Develop storage and retrieval system using Presto and ParquetORC Scripting with Apache Spark and data frame. Thanks and Regards K Padmaja Recruiter Email Gtalk 16801 Newburgh Road Suite112, Livonia, MI-48154,USA

Keywords: Softpath Technologies LLC, Oakland , Sr. Data Engineer, Other , Oakland, California