Teradata's capabilities make it ideal for handling complex, high-volume workloads, while Snowflake's flexibility offers a scalable cloud solution.

(EMAILWIRE.COM, October 31, 2024 ) The global Big Data and Data Engineering Services market is experiencing significant growth, driven by the increasing volume of unstructured data and the need for advanced analytics. Key factors driving this growth include the rise of IoT devices, social media,...

Aim/Purpose: This paper investigates the changing paradigms for technical skills that are needed by Data Engineers in 2018. Background: A decade ago, data engineers needed technical skills for Relational Database Management Systems (RDBMS), such as Oracle and Microsoft SQL Server. With the advent of Hadoop and NoSQL Databases in recent years, Data Engineers require new skills to support the large distributed datastores (Big Data) that currently exist. Job demand for Data Scientists and Data Engineers has increased over the last five years. Methodology: This research methodology leveraged the Pig programming language that used MapReduce software located on the Amazon Web Services (AWS) Cloud. Data was collected from 100 Indeed.com job advertisements during July of 2017 and then was uploaded to the AWS Cloud. Using MapReduce, phrases/words were counted and then sorted. The sorted phrase / word counts were then leveraged to create the list of the 20 top skills needed by a Data Engineer based on the job advertisements. This list was compared to the 20 top skills for a Data Engineer presented by Stitch that surveyed 6,500 Data Engineers in 2016. Contribution: This paper presents a list of the 20 top technical skills required by a Data Engineer.

Nick Singleton, MBA, is a seasoned leadership and business strategy expert in his leading role at Integra Connect LLC

White Plains, NY United States - Atlas Air, Inc. seeks a Data Engineer (White Plains, NY).Job Duties: Develop dynamic reporting models to support business and customer needs for real time, on-demand reporting analyze the On Time Performance (OTP) of each business type and customer network through th... View

IEEE / Institute of Electrical and Electronics Engineers Incorporated

IEEE Computer Society

This is a direct hire to my team, which is the client-facing arm of Superconductive. We're a really great company - a distributed team of kind, smart, interesting people who treat each other like human beings. Full job description is below - feel free to reach out with questions!

Our mission

Superconductive’s mission is to revolutionize the speed and integrity of data collaboration. We are the team behind Great Expectations, the leading open source tool for defeating pipeline debt through data testing, documentation, and profiling. Data teams all over the world use Great Expectations to boost the confidence, integrity, and speed of their work.


About the role

As a Data Engineer on Superconductive’s Enterprise Partnerships team, you will directly contribute to the success of our partnerships with enterprise clients. You will be responsible for owning the end-to-end process of designing, building, and deploying data pipelines, using Great Expectations for data validation and testing. As part of this, you will be exposed to a wide variety of clients, industries, and types of data infrastructure.


You will be expected to:

Develop a deep understanding of client needs and goals, as well as their existing data infrastructure
Design, build, and deploy different pieces of data pipeline infrastructure, such as tools for ETL, data warehousing, and workflow orchestration
Train our clients in best practices for building, maintaining, and validating their data pipelines
Synthesize learnings from client work and identify high value product improvements to feed back into Great Expectations


As part of Superconductive, you will also have opportunities to evangelize Great Expectations through conference talks, workshops, and blog posts, if you choose to do so.


Why join Superconductive?

Glad you asked!


We offer…

A world class team, with deep roots in open source, cutting-edge software and data development. We are actively cultivating a new cultural blend of excellent data engineering and AI-enabled technical workflows.

A fast-growing company with lots of opportunity for learning and personal growth.

A front-row seat to the rapid evolution of data science and engineering. Data work is going through a renaissance, and—as the leading provider of a key piece in the new technology ecosystem—Superconductive is right in the middle of it.

A kind, curious, and open-minded company culture. We are always seeking ways to improve ourselves and our processes; we keep these conversations open to the whole team. We prioritize empowering our team members rather than a command and control hierarchy.

A distributed team with lots of flexibility around timing and individual work preferences. We currently have teammates in the San Francisco Bay Area, Salt Lake City, New York, Michigan, North Carolina, and other places. We’d love to add your town.

And of course, competitive compensation (base salary + equity package) with available Medical, Dental, Vision & 401K.


Job requirements

Minimum two years of work experience as a data engineer or in a role with responsibilities including:
Design, implementation, and maintenance/management of data pipelines
Implementation and management of SQL databases

Proficiency in SQL and Python

Proficiency in engineering tools and best practices, such as version control and testing

Experience with data analytics and relevant tooling (e.g. Python data stack, R)

An excitement for interacting with clients and digging deep into new (to you) technologies and domains


Strong candidates for this role will most likely come from backgrounds in either data engineering or data science. We are looking for people with substantial experience building data pipelines, or experience developing software with a close affinity to data and analytics.


Great engineering is more than just technical proficiency. Our team is set up to support and build the following cultural skills. We don’t expect black-belt virtuosity. We do expect everyone to bring good ideas and habits, so that they can contribute to a collective culture of best practices.


Have data in your blood - the more curiosity you have about a wide variety of data from all walks of life, the better.

Communicate well—in writing, in person, and on Slack.

Be good at time management (e.g. estimating timelines, prioritizing tasks, avoiding rabbit holes, speaking up when you need help).

Possess keen antennae for the lived experience of technical work. We constantly seek out opportunities for delightful refactors to tools, workflows, and collaboration, gathering ideas from all team members.

Commit to a culture of transparency, cooperation, and mutual support in an entirely distributed team.

Play nice with others. Assholes need not apply.


Nice to haves

Experience in a client-facing role, e.g. sales engineering, implementations, forward deployed engineering

Project management experience

Experience with domain-specific data in multiple industries (e.g. healthcare, ad-tech, e-commerce)

Advanced skills in data analytics, statistics, or machine learning

Devops and data infrastructure skills are a plus (e.g. workflow orchestration tools, containerization)

Experience with software engineering, and possibly contributions to open source projects


Superconductive Health, Inc. is an Equal Opportunity Employer. In compliance with Federal law and the Federal Acquisition Regulation (FAR) 22.1800, the selected candidate will be required to provide documentation that will verify their identity and eligibility to work in the United States.

Company: Prosum Technology Services
Experience: 0 to 50
location: India
Ref: 24344566
Summary: At our company, data is central to measuring all aspects of the business, and critical to its operations and growth. The BI data engineering team is responsible for collecting, analyzing and distributing data using public cloud and....

Company: Mulya Consulting
Experience: 5 to 10
location: Delhi NCR, Delhi
Ref: 24818166
Summary: * * * Client: Large Indian Company with Revenue over 3 Billion USD * location:bangalore / Noida * * Azure skilled data engineer * ....

Company: Vibrant Hr Consultancy
Experience: 2 to 5
location: Coimbatore
Ref: 24828221
Summary: Greetings from vibrant HR Consultancy!!! We have immediate openings for Data Engineer Interstead candidate please share me your updated resume to [HIDDEN TEXT] OR Contact - 9343075322

Company: Vibrant Hr Consultancy
Experience: 2 to 8
location: Coimbatore
Ref: 24828148
Summary: Greetings from Vibrant HR Consultancy !!! We have immediate openings for Data Engineer. Interested candidates kindly share your updated resume to [HIDDEN TEXT] Contact no - 6366680486

Cham, Switzerland : Springer, [2019]

Cham, Switzerland : Springer, [2019]

IEEE / Institute of Electrical and Electronics Engineers Incorporated

£Attractive: Paul Scherrer Institute
For more latest jobs and jobs in Switzerland visit brightrecruits.com