компания "атыраутехинвест" role requirement & responsibilities: - design, develop, and maintain data integrations for data storage and data mining.- perform data warehouse design and testing, including data design, database\storage architecture, metadata, and repository creation.- design data warehouse schematics, layouts, and capacity planning.- create and implement effective metrics and monitoring processes.- create and optimize data catalog for existent- develop processes and procedures for entering information into data warehousing systems and for ensuring reliability of information entered.- seek continuous improvement in performance of data warehouse and ensure security of data.- conducts complex work important to the organization.- works under minimal supervision with wide latitude for independent judgment.- mentoring and providing guidance for data engineers- typically requires 2+ years of relevant experience or equivalent combination of experience and education.- data extraction from different sources. advising and enhancing guardrails for new data extraction methods- data modeling: design and implementation of data models for both integration and presentation repositories.- production automation: design and schedule the etl/elt process for periodic automated updates, including ftp/sftp file transfers. automated updаtеprocesses must have built-in fail-over strategies and must ensure data consistency and correctness.- t-sql programming: writing complex stored procedures, efficient sql queries with joins, and tuning for optimal performance. structured query language (sql), t-sqldata warehousing and etl\elt and analytics tools, e.g. ms tools(ssis, azure data factory)programming languages c#/pythonazure devops/agile/scrumknowledge and experience of working with azure data lake storage, gen 2knowledge and experience of working with azure data factory (and azure data pipelines) for data transformation and orchestrationknowledge and experience of working with azure databricks with an emphasis on data transformation and data cleansingknowledge and experience of working with azure sql database / azure synapse for data warehouse purposesdesired skills in new cloud tools deltalake, spark sql, polybase, scala 5/2 remote work
Адрес
Казахстан Атырау