Where does data mining succeed, and why?
As previously noted, I have a Computerworld column coming out next week on data mining. The heart of the column is an enumeration of markets where data mining applications were having genuine success. Before I sat down to actually write the column, my list went something like this:
- There’s a large set of “early warning” apps where text mining is being deployed. Many of those same apps are addressed by data mining of tabular data too – antifraud, to start with, and also warranty tracking and indeed most of the rest.
- Data mining has been huge in CRM.
- The use of data mining in manufacturing to do failure analysis, improve quality, etc. is really on the rise. This goes at least somewhat beyond what one could reasonably pigeonhole as “early warning.”
- Data mining plays a big role in the life sciences, and is being applied to a broad range of other sciences as well.
- Data mining is a huge part of R&D at search engine and antispam vendors.
| Categories: Analytic technologies, Data mining | 8 Comments |
Further information on data mining
My September Computerworld column (I’ll post a link, no sooner than September 11) is about data mining. As promised in that column, here are some links and guides to further work on the subject.
- I have posted extensively on text mining over on the Text Technologies blog.
- In particular, much of the column was based on a post in which I discussed “early warning” applications of text mining.
- The research was informed by a trip to the KDD 2006 conference, about which I’ve blogged separately.
- SAS is world’s biggest vendor of this stuff, so if you want to know what the applications are, you might want to start with their website.
| Categories: Analytic technologies, Data mining | 3 Comments |
KDD 2006 conference on data mining and knowledge discovery
I went to the KDD 2006 (Knowledge Discovery in Databases) conference in Philadelphia last week. It was an interesting, if weird experience. The conference had been billed to me as the place where all the world’s great data mining/KDD experts gather. This turns out to have been old news; the conference has apparently fallen off some the past 2-3 years. What are left are an academic conference and a small trade show that seem to be only loosely coupled. Here’s what I experienced at each.
Read more
| Categories: Analytic technologies, Data mining | 2 Comments |
SAP, the process company
The following is excerpted from an actual SAP email I received. Emphasis mine.
| Categories: Enterprise applications | Leave a Comment |
Why an actual peace treaty is essential in Lebanon
War is inevitably a terrible thing. This truth is repeatedly forgotten or disregarded, not least in the Middle East, most recently by Hizbollah and Israel alike, and perhaps by other parties influencing the Lebanese conflict as well.
But I am writing today, not about hatred and folly in general, but about a narrower point – namely, the need for an actual peace treaty in Lebanon, after decades of a formal state of war. Such a treaty is, in my judgment, essential for Lebanon’s economic future. And so it is essential for Israel’s security too, and by extension for the security of many other countries as well. For if Lebanon does not thrive — if the people of Lebanon lose hope — Lebanon will remain what it has been for three decades, an unstable and uncontrollable enemy of the Israeli state.
| Categories: Economic development, Public policy and privacy | Leave a Comment |
AOL’s IM content privacy policy
AOL’s privacy policy isn’t as bad as Microsoft’s, but it’s no joy either. “Rights” can be a slippery word, as partisans all over the political spectrum are apt to point out.
The contents of your online communications, as well as other information about you as an AOL Network user, may be accessed and disclosed in response to legal process (for example, a court order, search warrant or subpoena); in other circumstances in which AOL believes the AOL Network is being used in the commission of a crime; when we have a good faith belief that there is an emergency that poses a threat to the safety of you or another person; or when necessary either to protect the rights or property of AOL, the AOL Network or its affiliated providers, or for us to render the service you have requested.
| Categories: Online and mobile services, Privacy, censorship, and freedom, Public policy and privacy | Leave a Comment |
Microsoft’s non-privacy policy
I just went to download Microsoft Messenger, and reviewed the terms and conditions. The following is excerpted, emphasis mine.
We consider your use of the Service, including the content of your communications, to be private. We do not routinely monitor your communications or disclose information about your communications to anyone. However, we may monitor your communications and disclose information about you, including the content of your communications, if we consider it necessary to: (1) comply with the law or to respond to legal process; (2) ensure your compliance with this contract; or (3) protect the rights, property, or interests of Microsoft, its employees, its customers, or the public.
EDIT: I can’t find anything at all about content privacy on the Yahoo! Messenger privacy page.
| Categories: Online and mobile services, Privacy, censorship, and freedom, Public policy and privacy | Leave a Comment |
Would a Google PC succeed?
Richard Brandt asked me to look over his post on the oft-rumored possibility of a Google PC. I actually opined on this back in January, when the rumors were rife in connection with a supposed Wal-Mart sales/marketing agreement. I concluded that that would make a lot of sense for internet connectivity and student/homework uses (I didn’t consider work-at-home or gaming uses because that didn’t seem a good fit with Wal-Mart). The reasoning I came up with back then looks good in retrospect, with only minor tweaks (e.g., my new reason for not worrying about IE-only websites is the IE emulation capability in Firefox).
Richard, however, goes further, thinking that Google could succeed in PCs used mainly to run word processing, spreadsheets, etc.. His arguments include:
| Categories: Google, Hardware, Online and mobile services, Privacy, censorship, and freedom, Public policy and privacy, Software as a service | 7 Comments |
Scatterchat and Tor vs. The Great Firewall of China
Hacktivismo has just released Scatterchat, an IM tool intended to beat repressive regimes’ firewalls. Unlike other anonymizer types of tools that use Chinese repression mainly as a marketing hook, Scattershot seems to truly be focused on its stated goal. I haven’t figured out whether it does much clever other than leverage Tor, an anonymous network established by the Electronic Freedom Foundation to try to beat traffic analysis. This all sounds like a perfect example of what I’ve been calling for — technological creativity directed at beating technological repression.
Universal surveillance of vehicle movements
- Last December, Bruce Schneier documented the UK’s near-term intention to track the movement of every vehicle in the whole country, mile-by-mile. He further reported that there are (longer-term, of course) plans or hopes to track people’s personal movements, via face recognition and the like.
- Forrester Research apparently now reports (I found out via Ian Turvill) that pay-as-you-drive insurance in Europe is on the rise. This is a clever idea, for a variety of reasons, but it requires tracking almost every choice a driver makes.
- Meanwhile, cruder movement tracking is provided by electronic toll payment, something that is bound to skyrocket due to time savings, to pollution/congestion benefits, and eventually also to the benefits of time-of-day pricing, which can further reduce traffic congestion.
This is scary stuff. And we’re not going to wind up stopping it, even if we try. We can only hope to blunt its ill effects, by adopting new laws and legal principles that prevent misuse of data the government has already collected.