data processing
Manipulating raw facts
Data processing is the manipulation of raw information—collecting, organizing, calculating, storing, or transmitting it to produce useful output.
The term emerged with early computing but applies to any systematic handling of data: The company's data processing department handles payroll and inventory.
Often shortened to DP in technical contexts.
Processing rate over time
- latencyDelay in system response
- optimizationMaximizing efficiency or performance
- bottleneckConstriction limiting throughput
- cachingTemporary local storage
indexing · load balancing · parallelism · bandwidth · profiling · overhead · compaction · deduplication
Sequential task progression
- scriptingWriting executable commands
- schedulingAssigning times to events
- orchestrationScoring for instrumental ensemble
- pipelinesTubes for transporting fluids
ETL · macros · task automation · triggers · integration · queue management · event-driven · streaming
Collected numerical information
- patternsRecurring arrangements or designs
- insightsPenetrating understandings gained
- analysisMethodical study of parts
- correlationStatistical relationship measure
regression · forecasting · visualization · business intelligence · clustering · classification · inference · sampling
Grammatical structure extraction
- filteringSelective passage of elements
- sortingArranging into sequence
- aggregatingCollecting into a total
- cleaningRemoving dirt and mess
validation · normalization · formatting · encoding · conversion · mapping · merging · splitting
Open-source cluster framework
- cloud computingInternet-delivered digital infrastructure
- NoSQLSchema-free database systems
- scalabilityCapacity for growth
- clustersGrouped collections
petabytes · servers · database management · SQL · APIs · microservices · containerization · virtualization
throughput
Processing rate over time
- latencyDelay in system response
- optimizationMaximizing efficiency or performance
- bottleneckConstriction limiting throughput
caching · indexing · load balancing · parallelism · bandwidth · profiling · overhead · compaction · deduplication · prefetching
workflow
Sequential task progression
- scriptingWriting executable commands
- schedulingAssigning times to events
- orchestrationScoring for instrumental ensemble
pipelines · ETL · macros · task automation · triggers · integration · queue management · event-driven · streaming · transformation
statistics
Collected numerical information
- patternsRecurring arrangements or designs
- insightsPenetrating understandings gained
- analysisMethodical study of parts
correlation · regression · forecasting · visualization · business intelligence · clustering · classification · inference · sampling · charting
parsing
Grammatical structure extraction
- filteringSelective passage of elements
- sortingArranging into sequence
- aggregatingCollecting into a total
cleaning · validation · normalization · formatting · encoding · conversion · mapping · merging · splitting · deduplication
Hadoop
Open-source cluster framework
- cloud computingInternet-delivered digital infrastructure
- NoSQLSchema-free database systems
- scalabilityCapacity for growth
clusters · petabytes · servers · database management · SQL · APIs · microservices · containerization · virtualization · replication
- computingcomputer, computers, software, hardware, code
- processingrefining, purifying, conditioning, filtering, sorting
- tech industrysoftware, computers, semiconductors, hardware, Amazon
- data sciencealgorithm, machine learning, dataset, model, analytics
- IT systemssoftware, server, hardware, servers, databases
- digitalizationcomputerization, digitization, digital infrastructure, digital platforms, digital services
data processingdata storage
Retaining digital information
raw data · data retention · data collection · data capture · data preservation
data, process, processed, processing, processor, processors, reprocessed, reprocessing, processable