Year Year arrow
arrow-active-down-0
Publisher Publisher arrow
arrow-active-down-1
Journal
1
Journal arrow
arrow-active-down-2
Institution Institution arrow
arrow-active-down-3
Institution Country Institution Country arrow
arrow-active-down-4
Publication Type Publication Type arrow
arrow-active-down-5
Field Of Study Field Of Study arrow
arrow-active-down-6
Topics Topics arrow
arrow-active-down-7
Open Access Open Access arrow
arrow-active-down-8
Language Language arrow
arrow-active-down-9
Filter Icon Filter 1
Year Year arrow
arrow-active-down-0
Publisher Publisher arrow
arrow-active-down-1
Journal
1
Journal arrow
arrow-active-down-2
Institution Institution arrow
arrow-active-down-3
Institution Country Institution Country arrow
arrow-active-down-4
Publication Type Publication Type arrow
arrow-active-down-5
Field Of Study Field Of Study arrow
arrow-active-down-6
Topics Topics arrow
arrow-active-down-7
Open Access Open Access arrow
arrow-active-down-8
Language Language arrow
arrow-active-down-9
Filter Icon Filter 1
Export
Sort by: Relevance
  • Research Article
  • 10.34218/ijitdm_01_02_001
BUILDING ADAPTIVE DATA ENGINEERING PIPELINES A FRAMEWORK FOR DYNAMIC OPTIMIZATION
  • Dec 30, 2024
  • INTERNATIONAL JOURNAL OF INFORMATION TECHNOLOGY AND DECISION MAKING
  • Pavan Manukonda + 3 more

In data-intensive systems, the construction of adaptive data engineering pipelines is an essential effort. To build efficient and adaptable data pipelines, this framework offers dynamic optimization methods. The technique can adjust to changing workloads and data characteristics because it uses real-time feedback and automated decision-making procedures. Both efficiency and effectiveness in using resources are improved by this flexibility. This research makes a substantial addition to literature by illuminating data engineering ideas, implementation techniques, and real-world applications. To effectively manage today's data-intensive settings, adaptive data engineering pipelines are a must. Create adaptable and efficient pipelines with the help of this framework's dynamic optimization method. It uses automation and real-time feedback to adapt to changing workloads and data attributes, maximizing efficiency and performance. engineering pipelines is presented in this study. To build efficient and adaptable pipelines, the method places an emphasis on real-time feedback and automated decision-making. It improves efficiency and effectiveness by adjusting to new data types and workloads. This research provides an important addition to the development of data engineering techniques by outlining the fundamental ideas, methods of implementation, and possible applications. A methodology for building adaptive data engineering