The Enterprise Decision Support and Data Governance (EDGE) Innovation team is seeking a highly skilled Data Engineer who will also serve as a Data Subject Matter Expert (SME). In this role, you will play a crucial part in developing and maintaining the enterprise curation layer within the EDGE. You will leverage your extensive experience in data engineering to ensure the data within EDGE is accessible, reliable, and high performing for our utilization by our Innovation, data & analytics teams throughout the enterprise.
This position follows a hybrid work schedule. Employees are required to be in the office four (4) days per week.
This position may be filled as an Engineer I (GSE05) or Engineer II (GSE06). Individuals who demonstrate advanced skills in terms of work experience, education, and certifications may be eligible for the more advanced position.
SUMMARY
The Data Engineer will collaborate with business and technology stakeholders to understand requirements, define data models, and implement Gas data assets. They will manage data governance and maintain a data catalog. The role includes supporting machine learning model creation, training, operation, and automation. Additionally, they will enforce data standards and collaborate with various teams to deploy optimized data solutions, plan capacity, and ensure secure data storage and usage.
MAJOR JOB RESPONSIBILITIES:
Functional Expertise:
- Provide key knowledge, insight, and informational guidance related to the integration of data and design of the holistic data landscape in support of the EDGE organization's objectives
-
Translate Decision Support Requirements related to KPI's, Indices, and other analytical insights into Executable strategies, approaches and platform designs or other means of information delivery
-
Produce & Manage all Decision Support Solution Design Frameworks and their supporting Documentation
-
Identify and propose new capabilities and improvements to Decision Support capabilities
-
Guide, Advise and Support other teams on implementation issues related to application of data related capabilities and approaches
-
Perform expert oversight and guidance for delivery of assigned Decision Support Solutions
-
Support Business Information Architects in the communication and interpretation of information and analysis to business leadership
Responsibilities
-
Support EDGE Project related assignments as required
-
Coordinates with the TO to identify approaches and solutions to data implementation problems related to the supporting EDGE solutions platforms
-
Develop and enhance the data management processes within the curation layer of the EDGE platform, ensuring scalable and accurate data handling and retrieval.
-
Create data models and construct efficient data pipelines, ETL processes, and workflows using Databricks, optimizing data ingestion transformation, and storage to support analytical needs.
-
Collaborate with data scientist to refine data collections methods while building and tuning Databricks jobs and Apache Spark application for improved performance and scalability.
-
Implement best practices for data management practices including partitioning, indexing, and compression to increase query efficiency.
Business Acumen:
-
Evaluates and recommends approaches and strategies on deployment and application of the tools and technologies adopted by Southern Company with the TO in the development and delivery of solution strategy, platforms, and applications
-
Develop and strengthen data standards and policies to maintain data integrity and consistency across the EDGE.
-
Adhere to and implement data governance protocols to uphold high-security standards and clear data lineage.
-
Proactively monitor data systems to identify and resolve issues that could impact data accuracy.
Engagement:
-
Collaborates with other team members, TO and business managers to resolve complex business issues and problems
-
Develops and maintains productive working relationships with key EDGE and TO managers and staff
-
Engage with data scientists, analysts, and business stakeholders, to precisely understand their data needs and translate these into Databricks-based technical solutions.
-
Offer technical guidance and support, enabling the seamless introduction and adoption of advanced data solutions.
-
Remain abreast of the latest data engineering trends, particularly in Databricks and other cloud-based technologies.
Southern Company
30 Ivan Allen Jr Blvd NW
Atlanta
Georgia États-Unis
www.southerncompany.com
