Applications & Trends in Data Mining
How data mining is applied in four key domains: financial data analysis (warehouses, loan prediction, money laundering), retail (multidimensional analysis, campaign effectiveness), telecommunications (traffic analysis, fraud, sequential patterns) and biological data (integration, sequence alignment with BLAST/FASTA, protein networks).
1Overview
Earlier lessons covered the principles and methods of data mining. This lesson looks at application domains and how customised data mining tools should be built for each.
| Domain | Why data mining fits | Typical uses |
|---|---|---|
| Financial | Data is complete, reliable and high-quality | Data warehouses, loan payment prediction, credit policy, money-laundering detection |
| Retail | Vast data on sales, customers, transport and service, growing with e-commerce | Multidimensional analysis, sales-campaign effectiveness, customer retention |
| Telecommunication | Highly competitive, multidimensional call data | Traffic analysis, fraud detection, association and sequential patterns |
| Biological | Explosive growth of genomics and proteomics | Data integration, sequence alignment (BLAST, FASTA), protein networks |
2Financial data analysis
Banks and financial institutions offer banking services (checking and savings accounts), credit (business, mortgage and automobile loans), investment services (mutual funds), and sometimes insurance and stock investment.
Financial data is usually relatively complete, reliable and of high quality, which makes systematic analysis and mining easier.
| Case | What data mining does |
|---|---|
| Data warehouses for multidimensional analysis | Build warehouses for banking data and analyse its general properties multidimensionally |
| Loan payment prediction and credit policy | Attribute selection and attribute relevance ranking find the critical factors that influence loan repayment and credit rating, and remove irrelevant ones |
| Money laundering and financial crime | Integrate multiple databases (bank transactions, federal/state crime history), then use analysis tools to spot unusual patterns, e.g. large cash flows at certain periods by certain groups of customers |
3Retail industry
Retail collects vast amounts of data on sales, customer shopping history, goods transportation, consumption and service. It keeps growing because of e-commerce; some businesses such as Amazon exist only online, with no physical store.
What retail data mining helps with:
- Identify customer buying behaviour
- Discover shopping patterns and trends
- Improve customer service quality
- Achieve better customer retention and satisfaction
- Enhance goods consumption ratios
- Design more effective transportation and distribution policies
- Reduce the cost of business
| Case | What data mining does |
|---|---|
| Data warehouses | Retail data spans sales, customers, employees, transport, consumption and services, so there are many ways to design the warehouse |
| Multidimensional analysis of sales, customers, products, time and region | Timely information on needs, sales, trends and fashions; data cubes and visualisation tools |
| Sales-campaign effectiveness | Compare sales and transactions containing the promoted items during the campaign vs before or after it |
4Telecommunication industry
Telecoms grew from local and long-distance calls to fax, pager, cellular phone, internet messenger, images, e-mail and web data. Telecom, computer networks and the internet are integrating, and deregulation makes the market fast-expanding and highly competitive.
Data mining helps to understand the business, find telecom patterns, catch fraud, use resources better and improve service quality.
| Scenario | What data mining does |
|---|---|
| Multidimensional analysis | Call data is naturally multidimensional: calling time, duration, caller location, callee location, call type. Compare traffic, workload, resource usage, user-group behaviour and profit |
| Fraudulent and unusual patterns | Fraud costs millions of dollars a year (see below) |
| Association and sequential patterns | Find usage patterns for services by customer group, month and time of day to promote services |
- Identify potentially fraudulent users and their atypical usage patterns
- Detect attempts to gain fraudulent entry to customer accounts
- Discover unusual patterns needing attention, e.g. busy-hour frustrated call attempts, switch and route congestion, and periodic calls from badly programmed auto-dial equipment (like fax machines)
5Biological data analysis
The past decade saw explosive growth in genomics, proteomics, functional genomics and biomedical research: comparing the genomes of humans and other species, genetic networks, protein pathways, new drugs and cancer therapies. Biological data mining is now a core part of bioinformatics.
| Topic | Key points |
|---|---|
| Semantic integration of genomic and proteomic databases | Data comes from different labs and methods: distributed, heterogeneous and varied. Integration, plus linking research literature to biological entities, enables cross-site analysis |
| Alignment, indexing and comparison of nucleotide/protein sequences | Tools such as BLAST and FASTA. Unlike normal sequential pattern mining, they must allow gaps and mismatches to handle insertions, deletions and mutations |
| Structural patterns, genetic networks and protein pathways | Proteins fold into 3-D structures that interact by position and distance, forming complex networks. Find structural patterns and regularities |
For an applications question, name the domain, explain why its data suits mining, and give two or three concrete uses with the technique behind each. For example:
- Loan prediction: attribute relevance ranking
- Money laundering: integrate databases, then detect unusual patterns
- Telecom fraud: atypical usage patterns
- Biology: BLAST/FASTA alignment with gaps
6Quick revision sheet
| Domain | Remember it as |
|---|---|
| Finance: data | Complete, reliable, high quality |
| Finance: uses | Warehouses; loan prediction via attribute selection/relevance ranking; money laundering via integrated databases + unusual patterns |
| Retail: data | Vast; grows with e-commerce (Amazon is online-only) |
| Retail: uses | Buying behaviour, retention, distribution; data cubes by time/region; campaign effectiveness (during vs before/after) |
| Telecom: data | Multidimensional: time, duration, caller/callee location, type |
| Telecom: fraud | Fraudulent users, account break-ins, unusual patterns (busy-hour failures, congestion, auto-dialers) |
| Telecom: patterns | Association and sequential patterns by group, month, time of day |
| Biology | Bioinformatics; integrate heterogeneous data; BLAST/FASTA allow gaps and mismatches; 3-D protein networks |
?Self-check quiz
13 questions on this week. Answer them all, then press Check my answers. Your best and latest scores are saved and shown on the subject page.