Big Data In The Oil And Gas Industry: Opportunities And Challenges Essay.

Oil, referred to as 'Black Gold' has been an important resource to humankind for the last 3000 years but has been especially important since 1859 when the modern oil industry was born (Black, n.d.). However, in 2006, Clive Humby - British Data Scientist/Mathematician and architect of Tesco Clubcard, is credited with the phrase "Data is the new oil" (Arthur, 2013). In response to Humbly, Palmer (2006) argues that data by itself is not valuable - similarly to oil, it is "crude" until it is processed. This is supported by SAS (n.d.) who state that the emphasis is on processing and interpretation, ‘data can be analyzed for insights that lead to better decisions and strategic business moves’.

Write an essay arguing for or against the aforementioned title with a specific focus on one industry of your choice.

You are expected to use secondary research throughout your work to support your arguments.

• Define the term Big Data.
• Explain the characteristics of Big Data.
• Discuss legislation governing how data can be used or gathered.
• Choose onesector such as (but not limited to) retail, health, education, military, policing, intelligence.
• Explain how Big Data could is currently used in this sector and how it may be used in the future.
• Argue for or against the value of its use.
• Discuss ethical implications of this practice.

Characteristics of Big Data

Big data is the term which is used for the purpose of referring to the data sets which are large in size and are very much complex which the traditional application software are not capable of handling so as to process the data. Big data is the evolving terms which is associated with describing large volumes of structured along with the unstructured and semi-structured data which are having the potential of being mined in order to extract the information (George, Haas & Pentland, 2014). This are also used in machine learning projects as well as in advanced analytics applications.

The big data can also be characterized by making use of eth 3Vs which include the volume , the velocity and the wide variety. Later the big data as attributed with the concept of Veracity and Value. Data consisting of numerous cases are associated with offering greater statistical power whereas the data consisting of higher complexity might be associated with leading to a higher rate of false discovery. There are various challenges that are faced by big data.

The characteristics of big data has been listed below:

• Volume: The term big data is itself associated with describing the size, which is enormous. The data size is associated with playing a very vital in the process of determining the value out of a particular data. Besides this the volume of the data helps in determining if a data is a big data or not (Kitchin, 2014). For this reason, the volume is considered to be one of the major characteristics This characteristic is also to be considered while the big data are dealt with.
• Variety: This is the next characteristic of big data, which generally refers to the different heterogeneous sources and the nature of the data that is both the structures as well as the unstructured data. In early times the spreadsheets and the databases were the only sources of the data which were taken into account by most of the traditional applications. However now-a-days the data received from the emails, PDFs, photos and many more are being considered by the various analysis applications (Lv et al., 2015).  The variety of unstructured data is associated with posing certain issues.
• Velocity: This generally describes the speed at which the data gets generated. Besides this it also refers to the fact that how fast a data gets generated and how it is processed so as to meet the demands and is also responsible for determining the real potential that the data is having.
• Variability: Variability generally refers to the inconsistency that is shown by various data at times which is associated with hampering the processes related to the ability of handling and management of the data in an effective way.

Oil and gas organizations are associated with dealing with data of huge amount in last few decades for their mission to learn whatever lies below the surface and by making use of the big data it has become possible for the industry to gather and transmit the information in a more efficient way (Stephens et al., 2015). Oil and gas industry is also having the capability of leveraging the big data technologies so as to collect and manage the data from the rapid analysis of the seismic drilling and data of production. Besides this the big data usage also helps the industry in acquiring of new insights which would be helping in the improvement of the drilling process as well as the performance related to production.

Production of cost-effective energy has been made possible due to the advent of the digital oil fields along with addressing the various safety and environmental concerns. Various decision along with processes related to the exploration of the oil and natural gas are responsible for the development and production of huge amount of data and these data volumes grow at an exponential rate on a daily basis (Assunção et al., 2015).so the acquisition of the new data along with storage, processing of the solutions and development of new devices and many is likely to get doubled in next couple of years.

Oil and gas industry has been associated with dealing with huge amount of data for the purpose of learning what lies under the surface. Techniques like the data visualization, seismic software and many more. Besides this various new generation of universal computing devices are also used. This new generation of universal computing devices are associated with the usage of sensors which helps in the gathering and transmission of the data and this in turn is associated with opening new opportunities (Wamba et al., 2015). Oil industry has been associated with acknowledging the immediate breakthroughs and power which are to be found in the data by faster usage in a smart way.

Usage of Big Data in the Oil and Gas Industry

Advancements in the analytic capabilities as well as in the new tools have made the producers of oil and gas to become capable of capturing the more real-time data at low cost. This in turn would be associated with boosting up the oilfield and the performance of the plants by six to eight percentage (Chen & Zhang, 2014). The analytic capabilities that the oil and gas organizations are having for the purpose of competing in the industry needs to be improved which would be making the decisions to move faster and would be helping the stakes to grow even higher. A lot of time would be required for the purpose of building the world-class analytic capability and this would also be requiring a lot of investment as well (Popescu & Keller, 2016). This is only possible by having sustainable concentration of the higher management. Plans are to be developed which would be helping the organizations in mapping the ambitions of the company against the capabilities that they are having and the map that is to be prepared would be describing the path into an advanced world class analytic capability.

There are many oil and gas companies who are making attempts to use the Big Data technologies so as to solve the problems that they are facing. The application concept of big data in different functional areas of the corporate management can be applied. The areas mainly include the production area, logistic area, marketing area and many more. Big data is also having the capability of contributing towards making predictions related to the demand for oil products in the retail sales network (Chen, Mao & Liu, 2014). These predictions are used for the purpose of analyzing the prices and the changes taking place in the price by the competitors and the other regions. Due to the patterns that has been discovered, there are opportunities for having an increased sales of the products and the reducing delays taking place in the retail networks might also seem to be attractive.

Big data I also associated with enabling the process of analyzing the deposits of hydrocarbons, and the detection of the non-optimal areas of deposition helps in the selection of production programs and predicting the outcome. Engineers are having the experience of working with the various surrogated models processing and for this reason they are well aware of the various advantages of using eth analytic functions along with the physical models which are well aware of the various of using the analytic functions in addition to the physical models which makes use of the machine learning approaches (Raghupathi & Raghupathi, 2014). While considering the approaches to the data the production and the drilling generally reminds of the manufacturing which means that the engineers are having the capability of understanding the importance that the reality forecasting is having along with the surface and underground work modelling which is based upon the data that has been collected.

Particularly it can be stated that the Automation of the management of the entire drilling process would be requiring an in depth analysis of the data in a mode which is almost close to real-time. There exists huge amount of data in this field and the predictive analysis which is dependent upon them is very much important. Big data technology is also helpful in the production process. The systems are capable of monitoring the drilling process and is also capable of recording any kind of gas leaks, passage of water, change in pressure and any other type other type of changes which are the results of any seismic activity. Taking assistance form the information the engineers are capable investing and controlling the incidents which are likely to damage the equipment (Gandomi & Haider, 2015). Analysis have been associated with showing the following facts that the usage of the big data analytics in the oil and gas industry is still existing at an experimental level and only few organizations are there which are trying to use the Big Data technology.

Future of Big Data in Oil and Gas Industry and Current Status

Besides this the area of big data in this particular industry is still present at the early stage and at present the companies are associated with experiencing the usage of tools so as to tackle the projects. Efforts are being made for the purpose of testing the technology that are in practice and to access the potential benefits as well (Wu et al., 2014). Interpretation states that the massive advent of the Big Data in the oil and gas industry might not occur. Besides this the big data projects in this particular industry might not be developed, however there are certain IT tools which might arise and use the Big data.

The drastic changes that are taking place in the oil and gas industry would be combined with the advances made in the way by which the oil and gas industries are associated with the collection of the data so as to conduct huge range of exploration and production of oil. This in turn would turn out to be an existing time for the people who are working in the field of oil exploration (Hashem et al., 2015). Besides this the oil organizations are also capable of acquiring new insights which would be helping in boosting up the drilling process and the performance of the production while preventing the safety and the environmental concerns (Lazer et al., 2014). In addition to this the MapR could also be used so as to help the petroleum business to get capitalized on big data so as to optimize the operations, the lower costs and for boosting up the competitive edge.

The usage of the big data has been associated with spanning a wide array of fields. The ever-growing Big Data has been associated with posing a lot of new challenges which is not possible for the current methods of privacy self-management to deal with. The rapid growth of the information from the oil and gas industry has been associated with making it necessary to have an orchestrated effort to get involved in the research of understanding the various ethical challenges that are being faced, new policies are to be crafted for the challenges that are existing, along with the challenges that are likely to arise in the future (John Walker, 2014). Private always does not mean secret and ensuring the security and privacy of the data is not a matter related to the providing of definition and enforcement of the the information rules and this is not just a rule related to the collection of the data but is also about the ways of data retention and the ways by which the data is being used. Besides the requirement of transparency is another major aspect of the ethical implication.

References:

Assunção, M. D., Calheiros, R. N., Bianchi, S., Netto, M. A., & Buyya, R. (2015). Big Data computing and clouds: Trends and future directions. Journal of Parallel and Distributed Computing, 79, 3-15.

Chen, C. P., & Zhang, C. Y. (2014). Data-intensive applications, challenges, techniques and technologies: A survey on Big Data. Information Sciences, 275, 314-347.

Chen, M., Mao, S., & Liu, Y. (2014). Big data: A survey. Mobile networks and applications, 19(2), 171-209.

Gandomi, A., & Haider, M. (2015). Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management, 35(2), 137-144.

George, G., Haas, M. R., & Pentland, A. (2014). Big data and management.

Hashem, I. A. T., Yaqoob, I., Anuar, N. B., Mokhtar, S., Gani, A., & Khan, S. U. (2015). The rise of “big data” on cloud computing: Review and open research issues. Information systems, 47, 98-115.

John Walker, S. (2014). Big data: A revolution that will transform how we live, work, and think.

Kitchin, R. (2014). Big Data, new epistemologies and paradigm shifts. Big Data & Society, 1(1), 2053951714528481.

Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). The parable of Google Flu: traps in big data analysis. Science, 343(6176), 1203-1205.

Lv, Y., Duan, Y., Kang, W., Li, Z., & Wang, F. Y. (2015). Traffic flow prediction with big data: A deep learning approach. IEEE Trans. Intelligent Transportation Systems, 16(2), 865-873.

Popescu, M., & Keller, J. M. (2016, July). Random projections fuzzy k-nearest neighbor (RPFKNN) for big data classification. In Fuzzy Systems (FUZZ-IEEE), 2016 IEEE International Conference on (pp. 1813-1817). IEEE.

Raghupathi, W., & Raghupathi, V. (2014). Big data analytics in healthcare: promise and potential. Health information science and systems, 2(1), 3.

Stephens, Z. D., Lee, S. Y., Faghri, F., Campbell, R. H., Zhai, C., Efron, M. J., ... & Robinson, G. E. (2015). Big data: astronomical or genomical?. PLoS biology, 13(7), e1002195.

Wamba, S. F., Akter, S., Edwards, A., Chopin, G., & Gnanzou, D. (2015). How ‘big data’can make big impact: Findings from a systematic review and a longitudinal case study. International Journal of Production Economics, 165, 234-246.

Wu, X., Zhu, X., Wu, G. Q., & Ding, W. (2014). Data mining with big data. IEEE transactions on knowledge and data engineering, 26(1), 97-107.