big data
Analytics-scale information troves
Big data refers to datasets so large or complex that traditional software struggles to process them—typically measured in terabytes or petabytes.
The term also encompasses the technologies and methods used to analyze such data: Big data is transforming how companies understand their customers.
Often invoked loosely to mean any data-driven approach, sometimes with privacy concerns attached.
Algorithms that improve from data
- statisticsCollected numerical information
- business intelligenceData analysis for decisions
- regressionReturn to former state
- classificationSystematic grouping by traits
clustering · pattern recognition · decision trees · neural networks · algorithm · visualization · dashboards · metrics
Huge data measurement unit
- terabytesTrillion-byte units
- exabytesQuintillion-byte units
- massiveExtremely large and heavy
- velocityDirected rate of motion
variety · veracity · structured · unstructured · semi-structured · real-time · datasets · records
Open-source cluster framework
- cloud storageOnline data preservation service
- serversMachines providing network resources
- NoSQLSchema-free database systems
- scalabilityCapacity for growth
parallel processing · clusters · AWS · Azure · Kafka · streaming · batch processing · pipelines
Removing identifying information
- GDPREU data protection law
- complianceAdhering to requirements
- consentVoluntary agreement given
- surveillanceSystematic observation of subjects
encryption · security · regulation · transparency · accountability · opt-out · cookies · tracking
Uncovering deceptive schemes
- personalizationTailoring to individual needs
- targeted advertisingPromotions based on user data
- genomicsStudy of entire genetic sequences
- precision medicineTreatment based on individual characteristics
smart cities · risk assessment · credit scoring · social media monitoring · financial modeling · cybersecurity
machine learning
Algorithms that improve from data
- statisticsCollected numerical information
- business intelligenceData analysis for decisions
- regressionReturn to former state
classification · clustering · pattern recognition · decision trees · neural networks · algorithm · visualization · dashboards · metrics · KPIs
petabytes
Huge data measurement unit
velocity · variety · veracity · structured · unstructured · semi-structured · real-time · datasets · records · throughput
Hadoop
Open-source cluster framework
- cloud storageOnline data preservation service
- serversMachines providing network resources
- NoSQLSchema-free database systems
scalability · parallel processing · clusters · AWS · Azure · Kafka · streaming · batch processing · pipelines · Snowflake
anonymization
Removing identifying information
- GDPREU data protection law
- complianceAdhering to requirements
- consentVoluntary agreement given
surveillance · encryption · security · regulation · transparency · accountability · opt-out · cookies · tracking · Cambridge Analytica
fraud detection
Uncovering deceptive schemes
- personalizationTailoring to individual needs
- targeted advertisingPromotions based on user data
- genomicsStudy of entire genetic sequences
precision medicine · smart cities · risk assessment · credit scoring · social media monitoring · financial modeling · cybersecurity
- tech satireSilicon Valley, buzzword bingo, tech bro, startup culture, hustle culture
- data termsdatabase, record, CSV file, XML file, metadata
- technologyelectronics, computing, computer, internet, social media
- Tecnologiasoftware, hardware, internet, computadora, digital
- computingcomputer, computers, software, hardware, code
- Digital Ageapp, blog, emoji, hashtag, meme
big dataintuition
Instinctive immediate understanding
gut feeling · anecdotal evidence · case study · ethnography · heuristics
big dataspreadsheet
Tabular data organizer
structured data · relational database · flat file · tabular data · ledger