SCIGON is seeking a Senior Software Engineer (Data) for a contract to hire position in San Diego OR Campbell, CA.
Summary of Duties & Job Description
We are looking for a Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across teams in our group.
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.
- Implementing ETL process
- Monitoring performance and advising any necessary infrastructure changes
Skills and Qualifications
- Bachelor’s Degree or above (Computer Science, Bio Engineering, Electronics and Electrical Engineering or any related field)
- 5+ years of experience in data engineering
- Experience with building stream-processing systems, using solutions such as Spark-Streaming or Flink
- Experience with integration of data from multiple data sources
- Experience with building data lakes and data warehouses by leveraging any of the major cloud providers (GCP, AWS or Azure) is highly desirable
- Familiar with Hadoop ecosystem (HDFS, HBase etc.), especially Spark
- Good knowledge of Big Data querying tools
- Knowledge of various ETL techniques
- Knowledge of messaging systems, such as Kafka or RabbitMQ