The most blog postings from BeyeUK Copyright BeyeUK http://www.b-eye-network.co.uk/rss/all_blogs.php 144 24 http://www.b-eye-network.co.uk/images/logo_beyeblogs.gif http://www.b-eye-network.co.uk/rss/all_blogs.php
Relational database systems, such as IBM DB2 and Oracle Database, have undergone over a quarter century of development. During that time they have managed to successfully fight off competing database technologies for supporting mainstream database management. Do you remember the object/relational wars of the eighties? MapReduce, a software framework introduced by Google for supporting parallel processing over large petabyte files has garnered significant attention of late. IBM is experimenting with this in conjunction with Google, and GreenPlum recently announced support. The significant interest in MapReduce, and related technologies such as Hadoop and HDFS, has led to a backlash from the relational camp. David DeWitt and Michael Stonebraker have been especially outspoken (see www.databasecolumn.com/2008/01/mapreduce-a-major-step-back.html and www.databasecolumn.com/2008/01/mapreduce-continued.html). Here is a small quote from their thoughts on the topic: "As both educators and researchers, we are amazed at the hype that the MapReduce proponents have spread about how it represents a paradigm shift in the development of scalable, data-intensive applications. MapReduce may be a good idea for writing certain types of general-purpose computations, but to the database community, it is: 1. A giant step backward in the programming paradigm for large-scale data intensive applications 2. A sub-optimal implementation, in that it uses brute force instead of indexing 3. Not novel at all -- it represents a specific implementation of well known techniques developed nearly 25 years ago 4. Missing most of the features that are routinely included in current DBMS 5. Incompatible with all of the tools DBMS users have come to depend on" Does this mean the database wars are starting up again? My opinion is that MapReduce is not intended for general purpose commercial database processing and is therefore not a major threat to relational systems. However, it does have its uses (as Google has demonstrated) for certain types of high volume processing. It also demonstrates that as data volumes get bigger, and the complexity of data and data structures increases, other types of database technology may start to gain traction in certain niche marketplaces. The use by IBM of the SPADE language, instead of StreamSQL, in its InfoSphere Streams product (System S) also demonstrates the changes going on in the database market. What do you think?

]]> http://www.b-eye-network.co.uk/blogs/white/archives/2008/11/will_mapreduce.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Tue, 25 Nov 2008 16:20:38 MST http://www.b-eye-network.co.uk/blogs/white/archives/2008/11/will_mapreduce.php
Must be nice to wanted -- at least Mark Papermaster must be feeling the -- um -- love from his former and current employers. A federal judge ordered the former IBM executive recently hired by Apple to stop working for that company. I guess the big question remains... WHY?

]]> http://www.b-eye-network.co.uk/blogs/imhoff/archives/2008/11/something_rotte.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Mon, 10 Nov 2008 15:26:58 MST http://www.b-eye-network.co.uk/blogs/imhoff/archives/2008/11/something_rotte.php
I attended a breakfast forum this morning sponsored be Vista Ventures, an early stage venture capital firm here in Boulder, CO. The speaker was Ike Nassi, EVP for SAP Research. During his talk, Ike discussed "Green IT". He kicked off a whole series of thoughts in my head about how we can use BI to reduce our carbon footprints. So here goes:

]]> http://www.b-eye-network.co.uk/blogs/imhoff/archives/2008/10/green_it_it_is.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Fri, 31 Oct 2008 12:47:07 MST http://www.b-eye-network.co.uk/blogs/imhoff/archives/2008/10/green_it_it_is.php
The Apple iPhone has had a dramatic impact on the mobile phone industry, and the new Google G1 is also aiming to take a slice of this market. So far, however, these devices are targeted primarily at personal users, but the question is, “What role, if any, will these gadgets have in the business environment for applications such as mobile BI? At the recent SAP TechEd, I had the opportunity to address this topic with John Schwartz, CEO of Business Objects. I made the point that mobile computing was enjoying more success in Europe than the US, and asked him if he thought the Apple iPhone would change this. He agreed that mobile computing use was higher in Europe, but felt the majority of this was still for personal purposes. He noted that mobile BI was seeing growth in both the US and Europe. However, he said the device of choice for mobile BI was still the Blackberry, because its architecture was more suited to this type of processing. So far, he said, Business Objects has seen little demand for iPhone support. Donald Cambell, CTO of Cognos, took the same position when I asked him this question at a recent IBM Cognos analyst meeting. He said IT is still primarily supplying Backberry’s for business use, but in some cases IT will support personally purchased iPhones. He said the iPhone still doesn’t have the promised capability to run processes in the background, which limits its use for BI. He noted that the new Google G1 has an excellent development platform, and if the device is successful it could have a major impact on mobile business applications. He also made the interesting point that Windows Mobile was the preferred mobile platform for packaged embedded solutions used in locations such as hospitals, for example. I would be interested to hear from other people about how their organization is deploying mobile BI solutions.

]]> http://www.b-eye-network.co.uk/blogs/white/archives/2008/09/mobile_bi_do_th.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Tue, 30 Sep 2008 17:41:28 MST http://www.b-eye-network.co.uk/blogs/white/archives/2008/09/mobile_bi_do_th.php
Yep -- it's true. I took the summer off from blogging but now I'm baaaack! And what a time to come back. Dow down 777 points today. Banks and other financial institutions going under right and left. And the one area where there is huge growth turns out to be in the online fraud business. And sadly, these online fraud perpetrators are turning to software as a service (SaaS) to commit their fraud. What is the world coming to? Read on to see how SaaS is being used to commit acts of fraud.

]]> http://www.b-eye-network.co.uk/blogs/imhoff/archives/2008/09/summer_was_grea.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Mon, 29 Sep 2008 14:56:28 MST http://www.b-eye-network.co.uk/blogs/imhoff/archives/2008/09/summer_was_grea.php
As the worldwide banking crisis continues to escalate, one has to wonder—where was the Business Intelligence in all of this? What happened to Data Quality and Data Management? First, we had the interesting revelation that the individual banks and lending institutions all seemed to be blissfully unaware of the extent to which they were exposed by lending in the sub-prime mortgage market. It’s difficult to imagine how the information available to decision makers in these companies could have been so scarce or so uninformative. Most, if not all, financial institutions have had extensive and expensive data warehouses in place for many years now. Business Intelligence should easily have warned of the dangers. Was the increasing level of risk unmeasured, overlooked or simply ignored? More recently, we’ve had the spectacle of banks being unwilling to extend short-term lending facilities to one another for fear that the borrowing institutions could go belly-up in the next few days! Could the lenders not know? Unfortunately, in this case, the answer is probably that they couldn’t. Despite the fact that the worldwide financial market is tightly and instantly interconnected at a transaction level, the truth is that the underlying data remains disconnected and dispersed. Data Management and Data Quality have simply not been considered. Proper business governance in the financial markets as a whole is impossible without a well-defined and credible data foundation. So, assuming that we can survive the crisis without a meltdown, what has been happening should be a clarion call to Data Management professionals in the financial industry particularly but also beyond. We need to recognize the interconnected and increasingly fragile web of data dependencies that hold the business world together. It’s time to get out there and apply the principles we know and preach already. And we had better get moving quickly.

]]> http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/09/bi_and_the_financial_crisis.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Sun, 28 Sep 2008 19:52:30 MST http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/09/bi_and_the_financial_crisis.php
Well now - has the inevitable happened? Larry Ellison, Oracle's CEO finally recognised the value of the hardware/software combination in the busines analytics markeplace. This market, pioneered over 20 years ago by Teradata and now thriving with many other players including Netezza, ParAccel, Datupia, ExaSol, Vertica and others, has now become a target for Oracle who have clearly had enough of competitors eating away at the Oracle database with lower TCO DBMS offerings optimised for analysis and reporting. With the recent acquisition of DatAllegro by Microsoft, IBM with its Balanced Warehouse and now Oracle entering the database machine market it certainly seems that the DW Appliance market is now becoming a hot competitive battleground. The newly announced Oracle Exadata DW Appliance is jointly developed by Oracle and HP and will be sold directly by Oracle. The Exadata server runs the Oracle parallel server on Oracle Enterprise Linux. It has 8–HP Proliant DL360 G5 database servers, with • 2 quad-core Intel Xeon Processor E5430 (2.66GHz) • 32GB memory • 1–HP InfiniBand Dual Port HCA • 4–146GB SAS 10K hard disk drives 4–24-port InfiniBand switches 14–HP Exadata Storage Server Hardware—each is an HP ProLiant DL180 G5, with • 2 quad-core Intel Xeon Processor E5430 (2.66GHz) • 8GB memory • 1–HP InfiniBand Dual Port HCA • 12–300GB SAS or 12–1TB SATA disk drives My question on this announcement is given that HP are jointly in on the Exadata product offering with Oracle, what does this mean for HP's own DW Appliance offering - the HP NeoView Appliance? This is also a parallel DBMS product that competes with Oracle. I assume that with HP playing in both markets (its own DBMS product on its own hardware plus the hardware behind the Oracle Exadata offering) that it is seeking to maximising the revenue it can take by covering all bases. Time will tell. In my opinion it is clear that with so many vendors now in the DW Appliance market it is going to take a lot more than just TPC-H benchmarks to get a differential. It certainly means customers will have to look closely at performance claims. Everyone will claim they are the fastest which could easily result in prospective customers demanding more to distinguish one vendor from another. For this reason I believe that analytic application appliances have to happen (analytic application pre-installed on a DW Appliance). Vendors who go deep on vertical analytic application appliances could carve out a very lucrative business when you combine this with the attraction of low TCO DW Appliance offerings.

]]> http://www.b-eye-network.co.uk/blogs/ferguson/archives/2008/09/oracle_enters_the_dw_appliance.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Thu, 25 Sep 2008 14:17:05 MST http://www.b-eye-network.co.uk/blogs/ferguson/archives/2008/09/oracle_enters_the_dw_appliance.php
Enterprise BI shops and data quality departments regard spreadsheets largely as the work of the devil. Against all the rules of information quality, data in spreadsheets is manipulated by users at will and in private. Then the resulting data and function is distributed, shared and further played around with, until it's anybody's guess whether the results presented at the end bear any relationship to the truth. Data that was pure and clean as it came out of the data warehouse, data mart or approved BI report is now potentially as contaminated as nuclear waste. And yet, check in with the users. Indeed, check in with yourself. Why is Excel so popular? Because it makes it easy to play with the data, check out hypotheses, get answers otherwise unavailable, and so on. And once you've gotten the answer through the spreadsheet, chances are you won't get the time or the resources to recreate the process in a more auditable, quality-conscious way. It's a real and spreading problem. But, what to do? This week I had the opportunity to preview a new product called Lyza that's due to launch on Sept. 22. In fact, you can download it and play with it already. Scott Davis, the CEO of Lyzasoft Inc. explained that they had spent a lot of time investigating how business analysts, the power users of spreadsheets, actually work. This is usually a good idea, because you find out what the users really need, and which of your assumptions are right or wrong. It will probably come as no surprise that most analysts approach their work in a highly unstructured and iterative way, pulling bits of relevant data into Excel from a variety of known sources - both official data marts and reports as well as unofficial files, spreadsheets, etc. they happen to have created before or borrowed from trusted colleagues. And they do it in Excel, because that's the only way they can. What Lyza does is to provide an easier, more intuitive way of pulling data together from diverse sources, combining and manipulating it and creating results and reports for distribution to the business. Well, that's all fine and dandy for the business analysts you may say, but how does it help the BI and data quality departments address the data contagion? The answer is that Lyza tracks and saves an audit trail of every action and every step of the analysis process that the user is building as well as enabling snapshots of the results to be cached and preserved for posterity. Now the data quality folks are beginning to smile. And the BI department? Well, they're less sure: they like the added traceability, but this is still outside their comfortable data mart zone. However, we could look at it in a different way. We could imagine that Lyza provides a new type of data mart - a "playmart" - a sand box where power analysts can experiment with data and perform all sorts of analyses in a safe, well-managed environment. Now, if only we could evaluate the analysts' logic and productionalize those analyses and reports that are going to be reused and built upon in the future. Scott's initial answer was that you can certainly do all this within Lyza itself. But a bit of further probing convinced me that the metadata that Lyza stores to describe the analysis processes is probably sufficient to enable the creation of ETL scripts for your ETL engine of choice. This would certainly require further investigation and automation, but it seems like the bones of the idea are there. In this case, the playmart could address a set of business analysts' needs that have been long ignored by the BI departments and by BI vendors as well. The only real fly in the ointment is whether Lyza will be able to convince the spreadsheet jockeys to get off their current Excel rocking horse and jump on the bright new Lyza pony in the playmart! (And that sentence would work so much better if only Lyza had chosen a mustang for their logo rather than a gecko.)

]]> http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/09/reining_in_the_spreadsheets_in.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Fri, 19 Sep 2008 16:54:35 MST http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/09/reining_in_the_spreadsheets_in.php
In my last post, I shared some thoughts inspired by the Decision Intelligence article written by Claudia Imhoff and Colin White. There, I suggested we need to really begin to consider all information as a single resource for the whole business. This entails stepping beyond our traditional IT-bounded view of our systems and looking at them with a renewed business vision. If we do this, it will also quickly becomes clear that our view of process needs reworking too. Claudia and Colin have drawn a box on the left of their architecture picture that arises directly from the insight that operational BI really is a different beast from the traditional BI we've all known and loved over the past 20 years or so. When you deeply consider the implications of building an operational BI system, as Claudia and Colin clearly have, it becomes obvious that operational BI has many of the characteristics of traditional operational or transaction-processing systems. Therefore, from a systems architecture point of view, you put them in the same box, in this case called "Business process intelligence". There are also some differences, of course. The most important is how the business users interact with these two related types of system. The value proposition of operational BI is that human decision-making skills can improve operational processes. How? Well, there are two very distinct threads here. One is the proposition that we can apply advanced analytics technology automatically to parts of the operational process. Fraud detection is a good example. Applying advanced analytics on the fly to credit card transactions gives better detection of fraudulent transactions. Note that this type of operational BI is almost completely invisible to the business users: they see the results of more fraud detected or less false positives, but how that happened is both unknown and uninteresting. The second thread brings users very directly into the loop. Here, the operational BI technology is made part of the users' visible process. Business users are presented with decision support technology that displays trends or exceptions in near real-time data, so that they can potentially choose a different course of action to that embedded in the normal flow. In effect, business users get to change the business process on the fly, rather than doing little more than data input as was previously the case. Now, keeping this in mind, here's the million dollar question. What's the difference between an operational system and an informational system; how do you distinguish between an operational process and an informational process? In the good old days, it was easy! The operational side was nearly or actually real-time, dealt with individual transactions or data elements according to a predefined process where the users had minimal freedom to act intelligently. Informational systems, in contrast, were centered around users who were expected to make intelligent decisions based on historical data without any clear process to turn those decisions into action. So, what is the answer today? When we in BI start building operational BI and the operational world starts implementing adaptive SOA-based systems, the distinction between operational and informational more-or-less disappears. This puts operational BI and operational systems together in one box of the architecture. But the deeper and probably longer-term implications of this bold step have not been explicitly called out. In fact, these implications are obscured by the naming of the new architecture as "Decision intelligence", because the top level of this architecture is no longer confined to the world that was formerly BI; it actually becomes the single, common process or interface through which all business users will interact with the underlying IT systems. Is that scary? Absolutely! But it is a clear and logical consequence of the paths that BI and operational systems are currently on. It means that we in BI are no longer in total control of our destiny. But the same is true of the operational systems. And, although I've not covered it here, collaborative systems (e-mail, office support, etc) are also being drawn inexorably into the same converged path. It's time we all started to talk to one another! And that does imply that decision intelligence may be too narrow a term for us all to agree on. May I propose again the "Highly Evolved Business"?

]]> http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/09/decision_intelligence_further.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Sun, 14 Sep 2008 12:26:10 MST http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/09/decision_intelligence_further.php
Claudia Imhoff and Colin White have a lengthy history of insightful and provocative contributions to the development of Business Intelligence. Their recent article, Decision Intelligence, is no exception. Their thesis is that the IT support needed for decision-making, now known as "Business Intelligence", today extends far beyond the traditional domain of data warehousing and is in need of a new architecture and a new name - Decision Intelligence. I fully agree. I've been using the terms "Highly Evolved Business" and "Business Insight" over the past year or so to express exactly the same thought. Indeed, Claudia, Colin and I have discussed this whole idea already at length and are very much on the same page. But I hadn't seen their architecture picture before, and it gives me the opportunity to discuss the whole topic from a higher perspective in this and the next post. Under Decision intelligence, the architecture shows three vertical blocks called “Business process intelligence”, “Business data intelligence” and “Business content intelligence”. The meanings of these blocks are fairly obvious, but take a look at the linked article for a full explanation. My thought is that they are almost too obvious: they closely reflect our current arrangement of systems building blocks in the IT world. Let’s first examine the data and content blocks. Today, if you look at typical enterprise implementations, you will certainly see databases and separate content stores. You’ll also notice independent systems built upon these separate stores. But, if you step back from the storage and processing issues, it’s pretty difficult to distinguish between the two categories. Try explaining the difference to a business user! Take an example of a clinician who’s trying to make a treatment decision. She’s looking at a chest x-ray - content in our terms. And she’s also looking at the “structured data” that goes with it: this x-ray is of a 45 year old male, smoker of 20 cigarettes a day for the past 30 years who has been admitted with shortness of breath. Does she see unstructured content and structured data that must somehow be combined in her decision making? I’d argue not. She simply sees a set of information she’s using. And some of the old barriers between the storage of structured data and unstructured content are breaking down. Where is the EXIF data (structured metadata) of a photo stored? Yes, in the JPG file along with the unstructured content. Where do e-mail systems store the structured metadata about sender, subject, date sent, etc? Sure, in the database with the unstructured e-mail body content. I could make a similar argument about the lack of distinction between real-time data (or operational) data and historical (data warehouse) data. My point is that if we want to create a new vision for the future, we need to start seeing the world through non-IT eyes. It’s all information. It’s a single concept; a single category of “stuff”. And we in IT need to start creating the tools and methods that allow us to create, manage and make available all information in a coherent and consistent way. At a conceptual level, that has to be the goal and that should be our first pictorial representation. Keep that thought in mind. I’ll come back to next time when I look at the process side of the picture.

]]> http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/09/decision_intelligence.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Mon, 1 Sep 2008 10:51:21 MST http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/09/decision_intelligence.php
I was browsing through the blogs on B-eye-network.com this morning (Sunday - yeah, sad, I know) and came across two recent entries that spoiled my coffee. Given that I'm no fan of instant gratification (in IT anyway), I'm not going to give you links, so you have to work at finding them yourself. But the phrases that caught my eye were "Instant SOA", "Data marts in about an Hour" and "full EDW's with AS-IS star schemas in 2 weeks". Now I'm as fond of a shortcut as the next guy, but I've learned the the word "Instant" is not all goodness. When I've bought some instant Spaghetti Bolognese in the local supermarket I've found that the cost is a lot higher than the individual ingredients and the taste, well, leaves a lot to be desired. Sure, I saved some time when I got home, but did I get value for money? And did I end up with what I really wanted? So, why should I expect more from an Instant DW? "Caveat emptor" as the Romans used to say. Here are a few contra-indications for when instant gratification should not be expected in your next BI (or SOA) project:

1. The business users are not quite sure what they want.
   Most BI projects start with a vague set of requirements from the potential users. It's going to take some time to hone these down to a usable definition of data and query needs. In the meantime, maybe it's best to let the users continue to play with their instant Excel spreadsheets and look over their shoulders to see what they're doing.
2. Somebody forgot to document the meanings of the data in the source applications.
   This is the oldest metadata problem. If your data sources have not been properly described, an Instant DW is likely to be instantly dismissed as misleading and inaccurate. Do you want to go there?
3. Garbage in, garbage out. Or worse...
   If your ingredients (data sources) are contaminated with erroneous data, you're going to end up with a very sick business on your hands if you just take the Instant DW approach. Understanding and fixing dirty data is time-consuming, but mandatory.

It's all about quality time... or quality vs. time. If I bring home my instant Spaghetti Bolognese, I may get it on the table within a few minutes. But, if the kids won't eat it or, worse, throw up that night, I'd argue I've made the wrong trade-off between time and quality. You need to consider the same balance in a BI or SOA project. Now, I'm off to spend some quality time with my kids :-)

]]> http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/08/instant_gratification_vs_quali.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Sun, 24 Aug 2008 12:20:20 MST http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/08/instant_gratification_vs_quali.php
I was at the Business Object Summit this week in Boston, where the main emphasis was on linking strategy to execution and a seeming focus on the larger enterprises. All very SAP-inspired, I thought. And very insightful, especially if you're a large enterprise. There have been some comments in the blogs already on these topics. But it was a small conversation over lunch that caught my interest... Information OnDemand. No, not the annual IBM Conference in Las Vegas, in October. But a rather low key effort from Business Objects with a website to allow companies to access market data and incorporate it into their BI efforts. There's a definite growing interest these days in combining external data with the contents of the warehouse. But it does raise some concerns, not least about the reliability of the external data and how to create a valid semantic relationship between the two data sets. In the past, companies have addressed these concerns by obtaining key market and other external data from trusted sources like Dunn and Bradstreet, Reuters and others and then ensuring that such data entered the warehouse via a controlled feed designed by Information Architects who could match the two data sets correctly. After all, such external data is another information source for the warehouse and should be managed like any other. This method works well for large enterprises with a centrally-controlled approach to the warehouse. And where the value-add derived from or risks incurred by using this data are significant, this method is probably still required. But what if you are a small or medium enterprise? Or what if you really only want to do a couple of once off analyses? Shopping at the Information OnDemand website appears to be the answer! Here you can buy prebuilt, but customizable, reports combining your data with external market and financial data. You can buy one-time snapshots or subscribe for regular updates. For larger companies, this could provide a safe and cost-effective way of dipping their toe in the big ocean of external data. For smaller companies, it could be all they need. Sounds like a useful idea to me! The service has been available since September 2007, but I hadn't come across it before. Maybe there are some similar services I should know about, so please feel free to comment.

]]> http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/08/access_to_quality_external_dat.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Thu, 14 Aug 2008 17:14:23 MST http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/08/access_to_quality_external_dat.php
The last few weeks have seen some major changes in Microsoft BI. There was the Datallegro acquisition, the announcement that Bill Baker was leaving, and the release of SQL Server 2008. For me, the first two changes are likely to have the most impact. SQL Server 2008 is well covered on the Microsoft Web site and so I won't address this topic here. The acquisition of Datallegro created a significant amount of interest, and over the last two weeks I have discussed the acquisition with a wide range of people. Although opinions vary, some general consensus has emerged from those discussions. The first question is why did Microsoft acquire Datallegro? The answer is that Microsoft wants to market SQL Server as a solution for large-scale data warehousing, and to do this they need to compete with the main DBMS vendors who have massively parallel database products. The Microsoft research group in San Francisco was spearheading key efforts to expand SQL Server in this area, but with the tragic death of Jim Gray it would appear the group lost momentum. To regain this momentum, Microsoft felt it needed to purchase an MPP database appliance. Why Datallegro? There are a wide range of database appliances on the market, but some of them use proprietary hardware and software techniques, and many of them are tightly integrated into the underlying system. What Microsoft needed was a conventional relational DBMS appliance where they could quickly replace Linux by Windows, and the open source relational DBMS by SQL Server. A smaller number of appliances fit this requirement. Datallegro was one of them, and as we saw the ultimate winner. What has Microsoft bought? This is a more controversial question. The first thing to note is the purchase price. The rumor mill reports the number to be $275 million. From my perspective this is a staggering sum for a company with a limited track record in data warehousing and few customers. Many people I spoke to were appalled by the sum paid by Microsoft. The feeling is that Microsoft purchased a marketing position rather than any real technology. One shouldn't of course underestimate the power of this marketing position. Whereas I don't think IBM or Teradata will feel threatened by the acquisition, it does put Oracle in a difficult position. Oracle has spent years trying to sell its shared everything solution as a competitor to shared nothing MPP approaches for data warehousing. The result is that it is losing market share to competitors such as IBM, Teradata and appliance vendors in the high-end data warehousing sector. From a marketing perspective, Microsoft is now adding to this competitive pressure. Oracle now has the choice of eating humble pie and acquiring or building an MPP solution, or seeing the continuing erosion of its large-scale data warehousing market share as customers see the benefits of alternative solutions. My money is on Oracle burying its head in the sand and doing nothing. The remaining question is whether Microsoft will succeed in using the Datallegro acquisition to penetrate high-end data warehousing. Given that SQL Server 2008 has just shipped, it means that the next release of SQL Server will not be for about another three years. A lot can happen in that time. Stuart Frost, the CEO of Datallegro, disputes this three year figure in blogs he has written, but Microsoft has a more rigid development, test, and release cycle than startups like Datallegro do. Even if the SQL Server MPP capability can be put into an interim release, it is unlikely we will see anything for two years. Another issue concerns how Microsoft is managing its data warehousing development. It would appear that the BI team in Redmond has ceded Microsoft's data warehousing strategy to the SQL Server Group. In turn, the SQL Group has set up a data warehousing center of excellence at the Datallegro HQ in Southern California. Datallegro has limited data warehousing expertise, and for Redmond to have this remote group driving its data warehousing strategy appears very risky. This brings me to the departure of Bill Baker to become CTO of Visible Technologies. This is a major loss for Microsoft. Bill is a visionary, an incredible leader, and was the driving force behind putting Microsoft on the BI and data warehousing roadmap. Microsoft has now lost two key BI and data warehousing people, Jim Gray and Bill Baker. They are irreplaceable, and setting up a data warehousing center of excellence in Southern California doesn't come close to making up for this loss.

]]> http://www.b-eye-network.co.uk/blogs/white/archives/2008/08/major_changes_g.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Wed, 13 Aug 2008 18:57:47 MST http://www.b-eye-network.co.uk/blogs/white/archives/2008/08/major_changes_g.php
I came across an ad today for a Google Webcast on Universal Search for Business. It contained the phrase "As the volume of information inside enterprises explodes, most executives recognize the importance of a Google-like search solution for business content.", which set me wondering... A Google-like search solution for business content? What exactly does that mean? The phrase "Google-like search", of course, covers a multitude of marketing-speak, but let's assume that it includes the patented PageRank technology behind Google's Internet search success. Google itself describes PageRank as follows: "PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value." (http://en.wikipedia.org/w/index.php?title=PageRank&oldid=230400158 as of Aug. 7, 2008). A number of questions arise for me: Does an enterprise intranet usually have a vast link structure? Would business executives really consider the "democratic vote" of the organization as a valid indicator of a document's importance? Indeed, how democratic is the link structure in an intranet? Google, Wikipedia and many Web 2.0 systems have an underlying belief in James Surowiecki's concept of "the wisdom of crowds". Data warehousing, Business Intelligence and, indeed, all traditional IT development tend to put more faith in experts and their accumulated knowledge. In the BI world, I'm beginning to see some level of acceptance that the so-called experts do not have a monopoly on business knowledge. We see that there is a growing need to allow and, indeed, facilitate the feedback of knowledge that emerges on the fringes of the BI community (the front-line staff and first-line managers) back into the core of the warehouse for wider promulgation and reuse. But, to what extent does Google and the Web 2.0 community recognize that some knowledge is inherently more useful or valuable (although not necessarily "right") simply based on the authority of its source? And within the tighter and more closed confines of an enterprise, that not all the requirements for wise crowds are met? If not, we may see the many years of careful effort by data modelers and administrators, and information stewards overturned in the rush to Web 2.0. This would not be in anybody's interest. On the other hand, if I've made the wrong assumption about what "Google-like search" means... Anybody care to comment? Or maybe I'll find time to sign up for the webinar!

]]> http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/08/enterprise_search_and_bi.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Thu, 7 Aug 2008 20:17:23 MST http://www.b-eye-network.co.uk/blogs/devlin/archives/2008/08/enterprise_search_and_bi.php
Scott Humphrey's Pacific Northwest BI Summit is my favorite event of the year. This is not only because it is at the Weasku Inn on the beautiful Rogue River in Oregon (which is only a 40 minute drive from Ashland where I live), but also because the vendors, consultants, and analysts who attend the event come together and share ideas in a unique way that is unlike any other event I attend during the year. Marketing hype and vendor competition are forgotten and everyone has down to earth formal and informal discussions on the state of the industry and its likely direction. This year was the seventh year the event has been held and it surpassed even the excellence of previous summits. The four analysts and consultants (Jill Dyche, Claudia Imhoff, William McKnight, and myself) were joined by representatives from Composite Software, Dataflux, Eyeris, HP, IBM Cognos, Infocentricity, Microsoft, Paraccel, PivotLink, SAP Business Objects, Teradata, Xactly Corporation, and of course the BI Network. The informal discussions covered a wide range of topics from BI to politics! The acquisition of Datallegro by Microsoft had just happened and this was a big topic of discussion. Although views varied, several people expressed the opinion that Microsoft was really buying an enterprise marketing position (especially against Oracle), rather than any real technology. By this time the industry blogging machine was working overtime and several blogs had already reported the purchase price to be $275 million, which staggered everyone. Towards the end of the summit, news was leaking out that Bill Baker was leaving Microsoft and everyone agreed this was a tragic loss for the company. Microsoft is certainly going through some dramatic changes in the BI area. The formal discussions focused on Software as a Service BI (led by Claudia Imhoff), CRM (led by Jill Dyche), Operational BI (led by myself) and IT Leadership (led by William McKnight). The volume of information and discussion is too lengthy to report here, but the BI Network will be releasing a number of podcasts on some of the discussions in the near future. Podcasts with each of the vendors are already available. Some key points I got from these four sessions were: 1. There is considerable interest in SaaS BI by both vendors and customers. BI is being used not only by SMBs, but also groups within large organizations. SaaS BI is often used to get a project started and many companies would like to bring the project in house once it matures. Many people felt that the pay as your go model will gradually become the norm for both SaaS and on-premises solutions (as pointed out at the summit "on-premises" is correct English usage, but "on-premise" is not). Lastly, like in-house application packages in the past, SaaS companies and solutions will merge and be acquired to provide a set of application solutions, rather than remain as stand-alone silos. 2. CRM is going through a reemergence with companies focusing on micromarketing, social computing as a new CRM information source, and increased interest in master data management. 3. There was universal agreement that operational BI is a big growth area, but that the range of solutions and vendors both inside and outside BI is large and confusing. One point of discussion was the convergence of operational BI with business process management and complex event processing. Other discussions focused on the impact of operational BI being process driven, rather than data driven, and on whether BI is the best term to use to describe analytical and decision making solutions moving forward. 4. The discussion on IT leadership generated many different viewpoints. There was universal agreement that companies need to focus less on reducing IT costs, and more on recognizing IT as a essential business component of the organization, in the same way, for example, that HR is. There was also a lot of discussion about the need for IT to modernize its thinking, and create a more flexible governance environment to handle emerging technologies such as social computing. The summit offered several opportunities to enjoy the many tourist activities of the Rogue River. The highlight for me was a visit to the Wildlife Images animal rehabilitation and education center. Everyone fell in love with a badger called Nubs with the result that the group donated $1,000 to support Nubs and other animals at the center. I can't wait until next year!

]]> http://www.b-eye-network.co.uk/blogs/white/archives/2008/08/thoughts_from_t.php?frss=1&ua=CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Wed, 6 Aug 2008 10:26:53 MST http://www.b-eye-network.co.uk/blogs/white/archives/2008/08/thoughts_from_t.php