QlikView Product Tour Video!
Rags to Riches
Retail Intelligence Blog - 'Looking at Retail From the Outside In'
Wednesday, May 23, 2012
FORBES.COM
"Techonomy founder David Kirkpatrick speaks with QlikTech CEO Lars Björk about his company at QlikTech’s headquarters in Radnor, PA. Below is a transcript of their conversation."
"Techonomy founder David Kirkpatrick speaks with QlikTech CEO Lars Björk about his company at QlikTech’s headquarters in Radnor, PA. Below is a transcript of their conversation."
Kirkpatrick: So Lars, just describe what QlikView does and what it is first of all.
Björk: I think we do something that’s very common to all people in business and organizations: we help them make smarter decision of the oceans of data that they have. Either it’s in legacy systems or in databases; we make that understandable for the average user in an organization, so it’s easy to pull together information.
Kirkpatrick: So generically this is in the category of business intelligence software, but what is the secret sauce that you guys bring to that?
Björk: You would put it in the bucket of business intelligence, but I think we have created a broader and much larger bucket that we call business discovery. And our big differentiator is, rather than starting from the technology and develop something for the user, we start from the user: “How would you like to interact with information? How would you like to present that information to yourself?”
Kirkpatrick: So you operate in a marketplace where a lot of giants also have big interests, whether it’s IBM, Microsoft, Oracle, or other big companies. How does that feel to be a sort of small, scrappy insurgent in that marketplace?
Björk: I think the experience is slightly different today than it was some years ago when we were clearly considered to be an ankle biter and annoying to these big players. Now I think we are a respectable player in this field; we are out-growing the industry with a factor of 6 to 8 times every year. And we know that a lot of these big players have picked up the more traditional vendors in this space in recent years, but I think they represent the old paradigm in this. This is where you build everything from the center. It is static views of how we interrogate data. And the question that I ask customers is, do you run a static company or do you run a dynamic company? Well, of course “I run a dynamic company;” nobody wants to be static. And that’s what we play in. The tool is very agile, it’s very dynamic. And it resonates with the use case. But I also think there is another dimension to it. There is such a big need out there. There is room for both. So we very often coexist with these big players in our clients’ space.
Kirkpatrick: Give us an example of a typical problem that QlikView/QlikTech would solve. I always say both; QlikView is the product.
Björk: QlikView is the product. So a typical problem, a very common one that people ask themselves if you run a business is, of course, how much did we sell? How do we track sales? How have we been performing in our customer relationships? But it can be all the way to a surgeon today in a hospital, how can they benefit from twenty years of information around a subject matter to treat that child that they have in front of them the best? Or you could take a Peruvian fisherman in a village who decides, where do we go out to sea to fish today? And they use all the information around currents and other stats around fishing before they decide to go out. So the use case is so broad; if we spend some time, I think we can go through any kind of industry.
Kirkpatrick: Do you have fisherman customers?
Björk: Yes, we do.
Kirkpatrick: You do?
Kirkpatrick: You do?
Björk: We do, we have fisherman customers. We have police as customers. We have surgeons as customers. We have had armies as customers. We have secret services as customers. And so on and so on. So I think the use case is just lots of data, unclear how you get to the information around that that needs to be presented in a way where more people in an organization can make decisions. I think there’s another trend playing into this as well. We talked about consumerization. I think the democratizing of decision-making is also something that’s playing in here. A large organization is going to benefit tremendously from empowering more people to make smart decisions, rather than a few at the top.
Kirkpatrick: So you’re trying to really position for a world of collaboration?
Björk: Absolutely, QlikView 11 is well positioned, recently out in November.
Kirkpatrick: That’s your current product?
Björk: Yes, the current version of our product. And I have to think we’ve just scratched the surface of it, early days, but we’re going to see a lot more in that.
Kirkpatrick: Any other trends that you see as sort of macro trends?
Björk: Of course there are bigger macro trends—cloud, big data—that we tap into as well. Big data is a very broad term and we play into that as well.
Kirkpatrick: How do you play into big data?
Björk: In the sense that what big data represents, you can ask yourself, is it a relative term or is it an absolute term? And I think the difference is between the user or the use case. I think the more important question that most people don’t even ask and that’s the relative data, something that I can view their business or their organization on tomorrow. On the broader aspects of this, we are drowning in data in today’s world and it’s not going to get any better. So how can we mine all that data, make sense out of it? And then cloud I think is going to be just a broader and bigger platform for deploying.
Original article: http://www.forbes.com/sites/techonomy/2012/05/17/qliktech-ceo-lars-bjork-on-surfing-the-deluge-of-data/
Friday, May 11, 2012
Big Data for Retail is Flying Off the Shelves
Great article from Forbes.com regarding BIG DATA at Retail! "Today’s retail business is a real-time, information-driven enterprise."
By Byron Banks, Vice President, Product Marketing, SAP
The days of relying solely on POS data to determine pricing and manage inventory levels are long gone. The same goes for relying solely on newspaper ads and mailed coupons to attract customers.
Maintaining long-term, profitable customer relationships requires a constant two-way flow of information between retailers’ storefronts (web or physical) and their suppliers and distribution networks. Competition is fierce and plentiful. If retailer X doesn’t have a desired item in-stock and at a competitive price, then the website for retailer Y, Z and many others are just a mouse-click away, or directions to their nearest store are easily found through a phone’s mapping app.
The best retailers have been using new mobility capabilities, Big Data technologies and advances like in-memory computing to revamp their business processes. They’re accessing information sources that didn’t exist before or were too costly or complex to use to become more nimble, cost competitive and able to delight customers with outstanding selection and service.
Spotting “Window Shoppers”
A key challenge in retailing has always been detecting and measuring lost sales. The cash register is the final system of record for all successful transactions, but what about the missed opportunities? Who was in the store or website, and what did they look at and not buy? If a retailer can understand the actions that didn’t result in a sale, they’re more likely to know if they need to adjust product selection, pricing or some aspect of displaying and promoting their offerings.
This type of information has been very difficult to track, but Big Data technologies such as Hadoop and in-memory computing are ideally suited to collecting and analyzing unstructured data types like the web logs that show the movements of every customer though an internet storefront. Web traffic data can then be combined with existing business intelligence applications and sales data to provide new insights. For example, retailers can compare the volume of website traffic for a given product versus number of sales of that product. You’d expect a correlation between web traffic and sales – consumers find the product they want, then they buy it. If instead you find a lot of web traffic but few sales, something is amiss. It’s a signal to the retailer to keep the product (whereas in the past they may have discarded it due to low sales) and confirm the product is competitively priced and has a compelling and informative presentation, array of colors and sizes, and all other aspects that are required to incent the customer to make that final, and most important, step: the purchase.
Staying Ahead of Traffic
Retailers are also using Big Data to better utilize their distribution networks and delight customers with improved on-time deliveries. In the past, dispatchers with clipboards and two-way radios would monitor daily customer deliveries and come up with workarounds to deal with traffic congestion, weather, construction and last minute rush orders. The system’s success was highly dependent on the intuition and expertise of the dispatcher. Today there are ways to make dispatchers job’s easier and augment their decision making with information from real-time data sources.
And wait, there’s more to come. The holy grail of retail has been to anticipate what consumers need even before they realize they need it. There’s no better way to beat the competition than to make an attractive offer and get a customer’s business before they even realize they need your product, or consider evaluating alternatives. Take printer cartridges, for example. There’s nothing worse than having to print a boarding pass with the taxi waiting outside and realize you’re out of printer ink. Today, retailers of office supplies are able to track purchases of customers’ in-store credit cards and rewards cards and, based on purchase history, anticipate when a consumer might need to reorder a product. Today they can send out an email offer for printer cartridges as well as an accompanying promotion for paper, with a guaranteed delivery time of 24 hours. Similar techniques are being used by travel companies (Time for your annual vacation?) and auto dealers (Looks like your car is due for service.) as well as other consumer-facing organizations.
Moving forward, imagine a world where retailers can use these new data sources to expand their consumer intelligence base to include analysis of customers’ social environments and web-patterns to become even more relevant and anticipatory of needs and interests. For example, an office supply store could not only provide you with an offer on printer toner but also show you who in your social network “Liked” that product and what else they bought. Or, they might indicate that friends in your social network have gone to a vacation spot you are exploring, and that they had a great time – or not. Clearly there is a need to manage the privacy aspects of this, but if retailers can provide an appropriate “opt in” and ways for consumers to manage the flow of information that is shared and who sees, it could be a great opportunity for retailers to enhance customer service and loyalty by providing consumers with better advice and less impersonal spam and junk mail.
Find original article here
By Byron Banks, Vice President, Product Marketing, SAP
Today’s retail business is a real-time, information-driven enterprise. Every customer interaction and movement of a product through a distribution network is measured and used to refine pricing strategies, update inventory decisions and tailor customer incentives on websites, email and mobile devices.
Maintaining long-term, profitable customer relationships requires a constant two-way flow of information between retailers’ storefronts (web or physical) and their suppliers and distribution networks. Competition is fierce and plentiful. If retailer X doesn’t have a desired item in-stock and at a competitive price, then the website for retailer Y, Z and many others are just a mouse-click away, or directions to their nearest store are easily found through a phone’s mapping app.
The best retailers have been using new mobility capabilities, Big Data technologies and advances like in-memory computing to revamp their business processes. They’re accessing information sources that didn’t exist before or were too costly or complex to use to become more nimble, cost competitive and able to delight customers with outstanding selection and service.
Spotting “Window Shoppers”
A key challenge in retailing has always been detecting and measuring lost sales. The cash register is the final system of record for all successful transactions, but what about the missed opportunities? Who was in the store or website, and what did they look at and not buy? If a retailer can understand the actions that didn’t result in a sale, they’re more likely to know if they need to adjust product selection, pricing or some aspect of displaying and promoting their offerings.
This type of information has been very difficult to track, but Big Data technologies such as Hadoop and in-memory computing are ideally suited to collecting and analyzing unstructured data types like the web logs that show the movements of every customer though an internet storefront. Web traffic data can then be combined with existing business intelligence applications and sales data to provide new insights. For example, retailers can compare the volume of website traffic for a given product versus number of sales of that product. You’d expect a correlation between web traffic and sales – consumers find the product they want, then they buy it. If instead you find a lot of web traffic but few sales, something is amiss. It’s a signal to the retailer to keep the product (whereas in the past they may have discarded it due to low sales) and confirm the product is competitively priced and has a compelling and informative presentation, array of colors and sizes, and all other aspects that are required to incent the customer to make that final, and most important, step: the purchase.
What about physical stores? Is it possible to use the same techniques to better understand shopper behavior? The answer is becoming “yes.” Some of the leading-edge retailers are now using these new technologies to analyze video from their in-store camera systems and create mappings of customer foot traffic throughout the stores. This Big Data stream is then combined with sales data to create new applications that help optimize store layout planning product placement and uncover situations where consumer traffic (interest) doesn’t match expected sales and thus signals an issue that needs to be investigated.
Staying Ahead of Traffic
Retailers are also using Big Data to better utilize their distribution networks and delight customers with improved on-time deliveries. In the past, dispatchers with clipboards and two-way radios would monitor daily customer deliveries and come up with workarounds to deal with traffic congestion, weather, construction and last minute rush orders. The system’s success was highly dependent on the intuition and expertise of the dispatcher. Today there are ways to make dispatchers job’s easier and augment their decision making with information from real-time data sources.
Radio transmitters on every truck along with bar codes or RFIDs on each package are combined with real-time mapping and traffic information to allow dispatchers to better monitor and visualize the progress of every delivery. Literally in the first minutes of the day, after just one or two deliveries, a predictive analytics application is already providing the dispatcher with revised estimated delivery times for the remaining orders based on past delivery data and current real-time traffic data on every truck’s route. If a dispatcher predicts a truck will miss a delivery, they can take immediate corrective action, such as re-routing a delivery or rescheduling with a customer. This new ability to provide organizations with easy-to-use applications that map incoming customer orders, real-time traffic and current truck location information has allowed leading organizations to do a much better job of meeting customer expectations and ensuring high operational efficiency in their distribution network. An example of a world-class organization that has implemented a number of these techniques is the online grocer FreshDirect.
Moving forward, imagine a world where retailers can use these new data sources to expand their consumer intelligence base to include analysis of customers’ social environments and web-patterns to become even more relevant and anticipatory of needs and interests. For example, an office supply store could not only provide you with an offer on printer toner but also show you who in your social network “Liked” that product and what else they bought. Or, they might indicate that friends in your social network have gone to a vacation spot you are exploring, and that they had a great time – or not. Clearly there is a need to manage the privacy aspects of this, but if retailers can provide an appropriate “opt in” and ways for consumers to manage the flow of information that is shared and who sees, it could be a great opportunity for retailers to enhance customer service and loyalty by providing consumers with better advice and less impersonal spam and junk mail.
Find original article here
Tuesday, May 8, 2012
SKYPAD Analytics at Tory Burch- Apparel Magazine 2012 Top Innovator
"Managing relationships with our retail partners is the job of the company’s wholesale division--- and is a crucial component of the brand’s growth. Without insight into POS and transaction data from retail partners, “We were not able to tell how well our products are selling across their locations,” Said Mike Giresi CIO at Tory Burch.
To solve this problem, Tory Burch elected to utilize SKYPAD Analytics!
Click here to read the complete article
Click here to read the complete article
Monday, May 7, 2012
Sears Competes On Big Data and Loyalty Programs
Great article from FORBES.com on BIG Data at retail! Sears Holding. Original Article can be found here
Tom Groenfeldt - Forbes Contributor
Sears Holding is betting on its innovative rewards program to spur the company’s growth. It is a path that Walmart can’t match because it doesn’t have a loyalty program — its best hope of understanding customer identity is through customers who use its credit card. Sears has a very intensive big data program to drive customer loyalty; the sophistication surprised me and should interest investors. I interviewed Dr. Phil Shelley, CTO at Sears Holdings Corporations about Sears’ use of the open-source Hadoop analytics platform kind of technology behind this award-winning program. Shelley was a keynote speaker at the Fusion 2012 CEO-CIO Symposium in Madison earlier this year.
As Laura Heller notes in her post at Forbes:
Sears is doing amazing things with technology; it has a hit on its hand with its loyalty program. Retail trade magazines have focused on the loyalty program, the technology, and the significant investment which Sears has made as indicators of improved performance in the future.
In naming Sears Holdings the Master of Enterprise Loyalty (Global) COLLOQUY said the company recognized that it had great brand recognition including Sears, Kmart, Lands’ End, Craftsman, Kenmore and DieHard, but it lacked customer recognition.
“Their best customers didn’t know they were best customers… a risky situation in a love them or lose them retail environment…
“The solution was 3 years in the making, which included programming that would capture, analyze, and report on customer activity at an individual level, across all 4,000 locations. The ensuing information would enable Sears to provide the differential treatment and personalized attention their best customers deserved…
“Results: Sears achieved an active member base in the 8 digits, exceeding the projected 36 month membership target in 17 months. Member spend is now identifiable, enabling SHC to understand member purchase history, leading to more relevant and targeted communications and offers. And just as anticipated, members shop more often and spend more per transaction than non-members.”
In fact, participation in the loyalty program is up to 80 million members and counting, according to the Sears Holdings website.
Sears Holdings manages that huge amount of data with the latest technology; Shelley joined the company work work on the transition from legacy systems.
Shelley said that big data and Hadoop allow the company to keep all its customers’ history to the lowest level of detail — “every customer, every SKU, every store, every point of history for as long as you want to and do statistical analysis on it. With Hadoop there really isn’t any restriction.”
Sears has been a leader among non-Internet companies in making the most innovative uses of big data, he added.
“We are bleeding edge on a large scale, Some of the innovations are just amazing. Now you can put all your data in one place and achieve a single point of truth and use it at a granular level that was pretty much impossible before.”
The tools can be used across the company’s operations. Fraud detection was an early use case but big data is also used for supply chain optimization, personal pricing, promotions, tracking marketing campaigns and locating and pricing overstocked inventory for clearance.
“We are re-engineering an old legacy company to become a big data company.”
John Perrone, Sears VIP and Clubs Loyalty Marketing Manager, said the program isn’t just about building loyalty but driving business results, according to CustomerManagementIQ.com. The key, said Perrone, is more comprehensively evaluating the level of customer engagement.
“They’re giving up their email or their mobile phone number to interact with us, and as long as we, the brand and the loyalty program, reply back with relevant information and information in the channel that this customer prefers, it’s a very successful engagement and interaction with that customer.”
According to Perrone, maximizing multi-channel customer loyalty therefore banks on the value of the trade-off. If loyal customers are expected to convey worthwhile information on their consumption and advocate on behalf of the brand, they need to see a “return” on that loyalty.
To Shelley, getting the customer’s opt-in and delivering value in return are key to success.
“We [as people] want personalized experiences. They mean more to us and are more relevant. I hadn’t thought about it much until we were getting into that level of personalization because it wasn’t possible before these tools became available. Now you can get very personalized.”
Cimphoni’s Rick Davidson, co-chair at the conference, said many IT organizations are befuddled by big data, and Shelley agreed.
“Everywhere we interface with others, this is being led by business people and not by IT. IT managers are struggling to get their arms around the big data open-source movement around Hadoop when they have built on expensive proprietary data warehouses. The IT people are not ready for it; the skills are very, very different. I would guess only 20 to 30 percent of IT management at the meetings I attend have a decent knowledge of what Hadoop is.” Many can’t spell it, he added.
Luke Lonergan, CTO of Greenplum who also spoke at FUSION 2012 (See post below) speaks about Hadoop traps that companies fall into as they deploy big data solutions. Shelley said they are a result of a serious skills gap.
“The way you would load and manage data is profoundly different. It is very easy for almost any IT shop to stand up a Hadoop instance, even on the cloud. Getting a proof of concept is easy, but making a production scale operation and adding business value, that is where people fall over all the time. There are so many things to consider — data modeling, injections, constructing the data, how to write the query, how to consume the data and how to integrate with legacy systems. You have a whole host of questions. People do get befuddled and lost and you see them in a proof of concept for months or even years.”
The tools for using big data have been maturing, but only a few firms are ready for it. A major challenge is thinking of the questions to ask of the data.
“The imagination is more the limit than the technology, the first time in my career that has been true. People used to say they needed some information but it would take a week to run. Now that limit is gone; the limit is people’s imagination.”
In a letter to shareholders earlier this year, Sears Holding Chairman Eddie Lambert offered hope to shareholders who may have been disappointed with financial results.
“We have significantly grown our Shop Your Way Rewards program, improved our online and mobile platforms, and re-examined our overall technology infrastructure. We believe these investments are an important part of transforming Sears Holdings into a truly integrated retail company, focusing on customers first.”
Lambert further emphasized the importance of technology to the company’s success in announcing his choice to fill the CEO slot which had been an interim appointment for three years. He selected Louis J. (Lou) D’Ambrosio, a Harvard MBA with experience at IBM and Avaya.
“From the beginning of our CEO search, we were determined to find a leader with information and technology experience who could catalyze the transformation of our portfolio of businesses in the context of the evolution of the retail industry that is occurring more broadly,” Lampert wrote in his chairman’s letter that accompanied the release of fourth quarter results.”
This was no surprise to Davidson. He has said that companies will look for IT experience in a CEO just as they have looked for experience in sales or finance now.
“Going forward, I don’t think CEOs are going to be considered fully qualified unless they have some technology background, not necessarily application development, but something like running a large IT project.”
At the Fusion 2012 CEO-CIO Symposium, Shelley charmed the attendees by de-mystifying Hadoop with some help from the audience. Tearing a page from the program, he ripped some text-heavy content into thin strips which he asked participants to search and count the number of times a letter appeared.
Positioning Jody Franz of the Outlook Group as a traditional relational database, he pulled six men from the audience to be the servers in parallel processing and a seventh who acted as MapReduce and distributed the strips of paper and then collected the results in a fraction of the time it took Franz to count through her selection.
Hadoop, named after the toy elephant of the founder’s son, is open source, free and it will do practical work, Shelley told the audience.
“If you have ETL (Extract, Transform, Load — a way to move data from one system to another) jobs that run for hours, they cause problems because you have to wait for the ETL to finish before the data is usable.” Hadoop can run queries within seconds of ingesting data,and once it is installed, the data never has to move again. Heavy bath jobs, even in COBOL, that take 90 minutes to run on a mainframe can be done in Hadoop in two minutes. To read one terabyte, sort it, write it back and sort it again could take days, added Shelley. Yahoo did that last year across 4,000 nodes in 62 seconds.
Hadoop clusters are extremely robust since the data is duplicated across three nodes; if one fails,, work is directed to the other two and when the node recovers, the data is written back to it.
Tom Groenfeldt - Forbes Contributor
Sears Holding is betting on its innovative rewards program to spur the company’s growth. It is a path that Walmart can’t match because it doesn’t have a loyalty program — its best hope of understanding customer identity is through customers who use its credit card. Sears has a very intensive big data program to drive customer loyalty; the sophistication surprised me and should interest investors. I interviewed Dr. Phil Shelley, CTO at Sears Holdings Corporations about Sears’ use of the open-source Hadoop analytics platform kind of technology behind this award-winning program. Shelley was a keynote speaker at the Fusion 2012 CEO-CIO Symposium in Madison earlier this year.
As Laura Heller notes in her post at Forbes:
Sears is doing amazing things with technology; it has a hit on its hand with its loyalty program. Retail trade magazines have focused on the loyalty program, the technology, and the significant investment which Sears has made as indicators of improved performance in the future.
In naming Sears Holdings the Master of Enterprise Loyalty (Global) COLLOQUY said the company recognized that it had great brand recognition including Sears, Kmart, Lands’ End, Craftsman, Kenmore and DieHard, but it lacked customer recognition.
“Their best customers didn’t know they were best customers… a risky situation in a love them or lose them retail environment…
“The solution was 3 years in the making, which included programming that would capture, analyze, and report on customer activity at an individual level, across all 4,000 locations. The ensuing information would enable Sears to provide the differential treatment and personalized attention their best customers deserved…
“Results: Sears achieved an active member base in the 8 digits, exceeding the projected 36 month membership target in 17 months. Member spend is now identifiable, enabling SHC to understand member purchase history, leading to more relevant and targeted communications and offers. And just as anticipated, members shop more often and spend more per transaction than non-members.”
In fact, participation in the loyalty program is up to 80 million members and counting, according to the Sears Holdings website.
Sears Holdings manages that huge amount of data with the latest technology; Shelley joined the company work work on the transition from legacy systems.
Shelley said that big data and Hadoop allow the company to keep all its customers’ history to the lowest level of detail — “every customer, every SKU, every store, every point of history for as long as you want to and do statistical analysis on it. With Hadoop there really isn’t any restriction.”
Sears has been a leader among non-Internet companies in making the most innovative uses of big data, he added.
“We are bleeding edge on a large scale, Some of the innovations are just amazing. Now you can put all your data in one place and achieve a single point of truth and use it at a granular level that was pretty much impossible before.”
The tools can be used across the company’s operations. Fraud detection was an early use case but big data is also used for supply chain optimization, personal pricing, promotions, tracking marketing campaigns and locating and pricing overstocked inventory for clearance.
“We are re-engineering an old legacy company to become a big data company.”
John Perrone, Sears VIP and Clubs Loyalty Marketing Manager, said the program isn’t just about building loyalty but driving business results, according to CustomerManagementIQ.com. The key, said Perrone, is more comprehensively evaluating the level of customer engagement.
“They’re giving up their email or their mobile phone number to interact with us, and as long as we, the brand and the loyalty program, reply back with relevant information and information in the channel that this customer prefers, it’s a very successful engagement and interaction with that customer.”
According to Perrone, maximizing multi-channel customer loyalty therefore banks on the value of the trade-off. If loyal customers are expected to convey worthwhile information on their consumption and advocate on behalf of the brand, they need to see a “return” on that loyalty.
To Shelley, getting the customer’s opt-in and delivering value in return are key to success.
“We [as people] want personalized experiences. They mean more to us and are more relevant. I hadn’t thought about it much until we were getting into that level of personalization because it wasn’t possible before these tools became available. Now you can get very personalized.”
Cimphoni’s Rick Davidson, co-chair at the conference, said many IT organizations are befuddled by big data, and Shelley agreed.
“Everywhere we interface with others, this is being led by business people and not by IT. IT managers are struggling to get their arms around the big data open-source movement around Hadoop when they have built on expensive proprietary data warehouses. The IT people are not ready for it; the skills are very, very different. I would guess only 20 to 30 percent of IT management at the meetings I attend have a decent knowledge of what Hadoop is.” Many can’t spell it, he added.
Luke Lonergan, CTO of Greenplum who also spoke at FUSION 2012 (See post below) speaks about Hadoop traps that companies fall into as they deploy big data solutions. Shelley said they are a result of a serious skills gap.
“The way you would load and manage data is profoundly different. It is very easy for almost any IT shop to stand up a Hadoop instance, even on the cloud. Getting a proof of concept is easy, but making a production scale operation and adding business value, that is where people fall over all the time. There are so many things to consider — data modeling, injections, constructing the data, how to write the query, how to consume the data and how to integrate with legacy systems. You have a whole host of questions. People do get befuddled and lost and you see them in a proof of concept for months or even years.”
The tools for using big data have been maturing, but only a few firms are ready for it. A major challenge is thinking of the questions to ask of the data.
“The imagination is more the limit than the technology, the first time in my career that has been true. People used to say they needed some information but it would take a week to run. Now that limit is gone; the limit is people’s imagination.”
In a letter to shareholders earlier this year, Sears Holding Chairman Eddie Lambert offered hope to shareholders who may have been disappointed with financial results.
“We have significantly grown our Shop Your Way Rewards program, improved our online and mobile platforms, and re-examined our overall technology infrastructure. We believe these investments are an important part of transforming Sears Holdings into a truly integrated retail company, focusing on customers first.”
Lambert further emphasized the importance of technology to the company’s success in announcing his choice to fill the CEO slot which had been an interim appointment for three years. He selected Louis J. (Lou) D’Ambrosio, a Harvard MBA with experience at IBM and Avaya.
“From the beginning of our CEO search, we were determined to find a leader with information and technology experience who could catalyze the transformation of our portfolio of businesses in the context of the evolution of the retail industry that is occurring more broadly,” Lampert wrote in his chairman’s letter that accompanied the release of fourth quarter results.”
This was no surprise to Davidson. He has said that companies will look for IT experience in a CEO just as they have looked for experience in sales or finance now.
“Going forward, I don’t think CEOs are going to be considered fully qualified unless they have some technology background, not necessarily application development, but something like running a large IT project.”
At the Fusion 2012 CEO-CIO Symposium, Shelley charmed the attendees by de-mystifying Hadoop with some help from the audience. Tearing a page from the program, he ripped some text-heavy content into thin strips which he asked participants to search and count the number of times a letter appeared.
Positioning Jody Franz of the Outlook Group as a traditional relational database, he pulled six men from the audience to be the servers in parallel processing and a seventh who acted as MapReduce and distributed the strips of paper and then collected the results in a fraction of the time it took Franz to count through her selection.
Hadoop, named after the toy elephant of the founder’s son, is open source, free and it will do practical work, Shelley told the audience.
“If you have ETL (Extract, Transform, Load — a way to move data from one system to another) jobs that run for hours, they cause problems because you have to wait for the ETL to finish before the data is usable.” Hadoop can run queries within seconds of ingesting data,and once it is installed, the data never has to move again. Heavy bath jobs, even in COBOL, that take 90 minutes to run on a mainframe can be done in Hadoop in two minutes. To read one terabyte, sort it, write it back and sort it again could take days, added Shelley. Yahoo did that last year across 4,000 nodes in 62 seconds.
Hadoop clusters are extremely robust since the data is duplicated across three nodes; if one fails,, work is directed to the other two and when the node recovers, the data is written back to it.
Subscribe to:
Posts (Atom)

