SigmaWay Blog

Transport for London (TfL) data, collected through ticketing systems, censored vehicles, traffic signals, survey groups etc. is provided through open API's for 3rd party app developers. This data is then used to produce maps showing when and where people are traveling, and allowing analysis at the level of individual journeys by using Big Data. The key priority to initiate this data was to provide travel information which gives the routes customers use and to send travel updates to them. Thus Bernard Marr from Forbes in his article showed how big data played a big part in re-energizing London's transport network. Read more about this article at: http://www.content-loop.com/big-data-internet-things-improve-public-transport-london/

The present day drug traffickers are very proficient in using modern technology, manipulating it for obtaining new marketing opportunities for their drugs. These criminals are using digital networking to a large extent for distributional purposes via secure methods, provided by modern day technology, which go undetected.  Often investigators fail to identify the disturbance they must be looking for, whose data lies well concealed under suitable digital protection. Advanced crime analytics incorporates big data and advanced analytics along with crime science study to detect anomalies, often their sources and the associated people. Advanced crime analytics is also essential in controlling cross-border drug trafficking by not only protecting witness details but also speeding up the entire process. To know more follow the article by Craig Richardson (chief executive of the Wynyard Group) at:

http://www.canberratimes.com.au/comment/crime-analytics-software-proves-powerful-weapon-in-war-on-drugs-20150414-1mkp6r.html

Ski resorts now-a-days are using certain frequency recognition systems, whose data is used to upgrade skier's experience, in multiple ways. For example, the system has led to the replacement of the old paper system, which unnecessarily consumed a lot of excess time. Stats collected from individual skiers are used by programs, to get an idea about the number of lift rides taken, number of days spent in the slopes, height scaled etc. Skiers can also get rewards based on their performance as directed by the data, thus leading to a gaming experience, which is attracting more and more skiers to the slopes. Big data also ensures that the ski resorts can efficiently transfer proper information to the consumers using data management. Big data not only helps the resorts in predicting weather, with the right stats, but also helps in strategizing suitable marketing techniques. Read more at:

http://channels.theinnovationenterprise.com/articles/even-ski-resorts-are-benefiting-from-the-big-data-explosion

In business, the 'black swan' is a form of disruption, which is usually least expected by a firm. Again, 'ugly duckling' refers to the day to day disruptions, which are always expected, and the firm remains prepared for it, hence, the coining of the term.  Companies don't pay attention to these disruptions with the expectation that they will remain small and never grow up to be as big and disruptive as the 'black swan'. Majority of the companies have risk management departments who focus on regulations and the traditional risks, completely ignoring the extreme ('black swan') cases. The 'black swan' has the capability of even surpassing the traditional data analytics procedures. Hence, comes into the picture, big data and advanced analytics, which gathers newer insights even about the most uncommon and worst case scenarios that may happen in the near future. Armed with information, much beforehand, the companies can make themselves immune, even to the worst of the situations. Predictive analytics thus plays a major role in recognizing 'black swan' events, often from the data of past 'black swan' episodes. Read more at:

http://www.teradatamagazine.com/v15n02/Features/Ugly-Duckling-or-Black-Swan/

Big data combines unstructured data with structured data to produce a report which provides a complete view of the organization. This report helps in decision making. Big data provide real time or near real time data. According to Mark Shacklett (President of Transworld Data), big data analytics results have impacted corporate revenues, expenditure and customer satisfaction. Web-based analytics on e-tail users helps in generating more sales by assessing customer preferences. Network diagnostics toolset analyze network and machine-generated data and produce predictive reports. This tool set gives a real-time view of network traffic. Sensors placed on railways and tram tracks, help crew members to proactively repair or replace equipment and railway tracks before they fail. Read more at: http://www.techrepublic.com/article/4-ways-real-time-analytics-lead-to-competitive-advantages/

We have come across the word "Smart city" frequently in recent times and we will see it more frequently in coming years. What exactly is a Smart City and how is it created?  The idea of smart City is to embed the technologically advanced devices which are making the Internet of things a reality into our surroundings. Once the technology is hard wired with the infrastructure, we can interact with these devices using our smart phones or computers and also devices can interact among themselves. For example, a car will be guided to an empty parking space without any human intervention.

In order to realize these smart cities, software and hardware applications for the internet of things are being encouraged. Some of the applications planned for smart cities are intelligent street lighting systems to conserve energy, mapping energy use around the city to understand demand and mapping how people maximize use of bicycle and foot paths. People are also raising caution over this new technology as it will hurt privacy.Read more at:http://www.forbes.com/sites/bernardmarr/2015/05/19/how-big-data-and-the-internet-of-things-create-smarter-cities/2/

In the age of Big Data, the data tables are going to grow enormously in size. Millions of rows of data will be obtained. Computational techniques available now will no longer be efficient in analyzing and interpreting the data. Enter this June; MIT researchers will present a new algorithm that will reduce the size of the data tables by leaving out bunch of rows. The data/rows remaining will be the representative ones of the total data. This is called condensed data/matrix of original data/matrix. To know more about how to handle Big Data, follow:https://newsoffice.mit.edu/2015/algorithm-shrinks-big-data-0520

• Apache Hadoop- as software vendors are targeting the distributed storage and processing architecture, need for Hadoop is increasing 
• Apache Spark-The rapid rise of the in-memory stack is being extended as a faster and simpler alternative to MapReduce-style analytics.
• NoSQL-Databases like MongoDB and Couchbase are taking over jobs previously handled by monolithic SQL databases like Oracle and IBMDB2.
• Machine Learning and Data Mining- Big data pros who can harness machine learning technology to build and train predictive analytic apps are in high demand Statistical and Quantitative Analysis-Add in expertise with a statistical tool like R, SAS, Matlab, SPSS, or Stata is of high demand in today’s world. 
• SQL- SQL is still in demand for the next-generation of Hadoop-scale data warehouses. 
• Data Visualization-It has become most important in the job market. 
• Progamming Languages-Knowing programming languages like Java, C, Python, or Scala could give you the edge over other candidates. Read more at: 

http://www.datanami.com/2015/01/07/9-must-skills-land-top-big-data-jobs-2015/

Natural disasters unlike other man-made disasters are the most terrifying events in the world since they cannot be controlled. However, by using the power of big data, it is possible to help in disaster management. For this purpose big data can be used through crowdsourcing which can be achieved by the increasing use of social- networking in the present days. In case of earthquakes, instead of using dedicated sensors which are highly expensive one can use the almost similar sensors in smartphones through Wi-Fi-hotspot and GPS to collect data to create an overall picture. For this, infrastructure needs to be set up so that information can be uploaded from the affected areas so that the affected people can be tracked down. Moreover the maps created through crowdsourced collaboration helps to optimize the recovery process. Read more at :http://channels.theinnovationenterprise.com/articles/big-data-in-a-crisis

The Retail industry is very competitive. Retailers need to apply analytics to analyze consumer behavior and retain them. Predictive Analytics help retailers to predict the response of customers regarding new offer, discount or product. But barrier of culture and stage fright, stop them to apply big data analytics.

Leslie Dinham (Teredata) in her article "two ways retailers are overcoming barriers to analytics adoption," talks about solutions to these barriers or adoption blockers. They are:

Barrier 1# Culture is the culprit: Employees get rigid due to working in the same culture, performing same job or duties. They don’t want to change their decision making process and roles. It becomes difficult to apply data analytics in this culture. The solution to this problem could be informing employee about the benefits of using data analytics and provide necessary training.

Barrier 2# Stage Fright: Many times, retailers won’t get success while applying analytics in their organization because they won’t able to choose the right team, tool or technology, won’t able to integrate new analytical capabilities into operations or the culture of the organization is not innovative. Paying attention while applying analytics in these things can help organizations to successfully apply analytics.

To know more about these barriers and solution to them, read an article at: http://www.forbes.com/sites/teradata/2015/05/13/two-ways-retailers-are-overcoming-barriers-to-analytics-adoption/

The retail industry is B2C industry. In B2C industry, forecasting and planning future demand and supply is a very important function to improve operation's efficiency. But, consumer behavior is very unpredictable. To analyze this unpredictable behavior, retail stores need to analyze big data. In Consumer Goods Analytics Summit in Chicago, suggestions on applying Big Data Analytics in Retail Industry were discussed. Let’s have a look on some of them:

·        By using big data analytics try to find out actual problem and their solution.

·        Apply analytics in every possible way from making sales report to multi-structured data to understand and improve customer service.

·        Always Interpret big data.

·        Recruit persons who understand the value of data analytics. 

To know more about Big Data Analytics in Retail, read the article link “Are retailers organized for Analytics” by Gib Basset, (Consumer Goods and Retail Industry Principal with Oracle Corp) at: http://www.retailwire.com/news-article/18266/are-retailers-organized-for-analytics

A data lake is a storage repository that holds a vast amount of raw data in its native format until it is needed. According to Gartner, the advantage of Data lakes is: helps in addressing the old and new problem by providing the relevant set of data for analyzing the situation. Disadvantages are:

• Lack of data quality.
• Security and access control.
• Data Lake requires proper infrastructure.

But using purpose built cloud systems security, access control and scalability problem can be solved, but data quality is not good.
 To know more about Data Lake and its advantage and disadvantages, read an article Data Lakes: Emerging Pros and Cons by Joe Panettieri. Link: http://www.information-management.com/news/Big-Data-Lakes-Cloud-Computing-Analytics-10026889-1.html

According to Sujan Patel (Contributor), companies before analyzing big data, must understand the company's goals and mission. In a survey by Price Waterhouse Cooper, only 44% of companies feel that they have the right talent to capitalize big data. When any company chooses tool for data analytics, focus should be on team needs and solution and the team must know how to use that tool. Read more at: http://www.forbes.com/sites/sujanpatel/2015/04/22/how-fortune-500-companies-are-building-big-data-teams-and-how-startups-can-too/

According to Collen Jones (CEO of Content Science and co-founder of ContentWRX), big data help to identify new market opportunities and customers. But, most companies are facing problem with analyzing big data. Therefore, before analyzing big data, an organization also needs to analyze small data. • Understand your situation by collecting and analyzing the data.
• Interpreting the data you collected in a clear and compelling way
• Searching what to do next?
Collected Data should be focused on content and customers and should be accurate and reliable. And while searching what to do next try to find opportunities and threats.
To know more read at: http://www.cmo.com.au/article/573077/thinking-big-data-marketing-get-small-data-right-first/

According to Mary Shacklett (president of Transworld Data), organizations need big data and analytics capabilities for immediately transferring big data into actionable decisions. According to a Gartner September 2014 report, there is an increase in investment in big data by 64% from 2013. Jeff Kelley (Big data analytics analyst from Wikibon), says that “customers expect personalization when they visit websites, so companies need to develop analytical capabilities and in the long term apply real-time will grow as Internet of Things.” Preventive maintenance analytics can be developed, if data on the Internet to thing can be analyzed. To know more about real time analytics and Internet of Things read on:  http://www.techrepublic.com/article/surge-in-real-time-big-data-and-iot-analytics-is-changing-corporate-thinking/

Punk in Punk Analytics means working fearless and having an approach to ‘do-it-yourself way’. In the last few years with the rise of new technologies, we entered into the era of punk-style analytics. Some characteristics of punk analytics are:

• No barriers for information search, as you can download easily and free of cost.
• Mistakes are part of the process, but we can overcome by using analytics software and by having familiarity with the data.
• Fast and to the point is good.
• Idea should be transparent and it should have less processed time.
• Punk Analytics are basically concentrating on the issues of the moment, as it is the starting point for active exploration.
•   Working collaboratively is important to achieve a common goal.
To know more about punk analytics follow the article link of James Richardson (Business Analytics Strategist at Qlik): http://www.itproportal.com/2015/04/19/rise-punk-analytics/

The investment banks are now looking at how they can use big data to do what they do better, faster and more efficiently. Senior executives at the banks want to enhance how they use data to raise profitability, map out markets and company-wide exposures, and ultimately win more deals. Big data is also a fundamental element of risk-profiling for the banks, enabling data analysts to immediately assess the impact of the escalation in geopolitical risk on portfolios and their exposure to specific markets and asset classes. Specifically, banks have now built systems that will map out market-shaping past events in order to identify future patterns. There lies the requirement of big data talent!

The banks are actively recruiting big data and analytics specialists to fill two main, but significantly different roles: big data engineers and data scientists. Data Scientists are responsible for bridging the gap between data analytics and business decision-making, capable of translating complex data into key strategy insight, while Data Engineers typically come from a strong IT development or coding background and are responsible for designing data platforms and applications. The competition between banks and fund managers to hire big data specialists is heating up. Data scientists are expected to have sharp technical and quantitative skills. They are in highest demand and this is where the biggest skill shortage exists.

To read more, visit the link given below:

http://www.computerweekly.com/opinion/Investment-banks-recruit-for-the-rise-of-big-data-analytics

The National Hockey League (NHL) is focusing on making a substantial investment in analytics to evaluate players' performance. Other than ticket pricing, analytics can be applied in other revenue stream like sponsorship investments by corporate partners. Using analytics in sports sponsorship provides answers to sponsors like why they are getting value by working with a sports organization. It also helps to state issues when a team or athlete is unsuccessful in competition. Thus, a wealth of new data is available to sports organizations. To know more, go through the article by Adam Grossman, Ben Shields, Irving Rein (authors of The Sports Strategist: Developing Leaders for a High-Performance Industry) at: http://blog.oup.com/2014/10/advancement-sports-analytics-business/

According to John Sculley (entrepreneur and the former CEO of Pepsi-Cola and Apple) a drastic change is taking place in marketing with the expansion of incredible new technologies like big data, cloud computing and numerous miniature sensors as well as mobile devices, making customers smarter. Nowadays customers value each other's opinion more than the producers'. With the beginning of a big-data marketing era, the prospects for entrepreneurs to disrupt and build companies have improved because technology costs have reduced. Read more at: http://www.entrepreneur.com/article/238363

Finding balance between data size and analytical modeling needs can be a problematic process in big data analytics. But it is essential for the success of big data analytics projects. The size of big data sources at social networking companies can obstruct analytics efforts. So, restraining the opportunity of an analysis by using data sampling techniques can be helpful. On the other hand, there are many businesses that lack the data needed to answer key business questions. For those organizations, development of effective analytical models can only take place after new data types are acquired and their analytics infrastructure is built. To know more, go through the article by Ed Burns (site editor of SearchBusinessAnalytics) at: http://searchbusinessanalytics.techtarget.com/feature/Balance-required-between-big-data-volume-analytics-needs