Managing the Data Avalanche

Models built on mountains of data can be impressive, but they can slide into disaster unless a company has the tools, training and data literacy to handle the challenges of digitalization

By Russell A. Carter, Contributing Editor

A phrase heard often lately — “mining is a data-driven industry” — might surprise someone unfamiliar with modern mining, and possibly even some who are. But, it’s not a stretch of the imagination to grasp that mining companies depend on access to many types of geological, operational, environmental and financial data to build and measure the effectiveness of their business plans. Mineral producers are literally sitting on a gold mine of knowledge, and just as they would with conventional mining operations, they need to find the best method for extracting maximum value from their digital resources.

Here’s how Arun Narayanan, group chief data officer for Anglo American, described it during the company’s technical and innovation update for analysts in May, as part of his description of VOXEL, Anglo American’s digital transformation platform for data-driven decisions: “We are taking a stand on data ownership. The data that we generate as part of our operations is an asset. It is our asset and we have to treat it as you would treat any other asset. We need to protect it, we need to make sure we are using it where it needs to be used and we need to make sure that we are getting the best return that we can get from that asset. And that is what the VOXEL data program sets out to do. It gives us a pathway to begin to claim ownership of that data and begin to use that data to drive the best decisions.”

Actually, mining has always been a profit-driven industry; it’s just now more apparent that data has become the fuel powering the industry’s drive to maximize revenue and reduce production costs. And, in today’s business environment, any mention of being data-driven is likely to be accompanied by at least one or more other terms that are tightly coupled to the concept, such as visualization, autonomy, predictive analytics, structured versus unstructured data and, of course, artificial intelligence.

Surveys indicate that two emerging issues — data management and security — also are rising in the list of concerns currently on the minds of mining executives, with about four out of five deeming it a critical item. Other studies illustrate how hard it is to find workable solutions: Mobile satellite communications provider Inmarsat said it found that the majority of mining organizations are struggling to meet security challenges presented by the Internet of Things (IoT) as they look for ways to harness the technology. Nicholas Prevost, director of mining at Inmarsat, said, “Our research clearly showed that mining businesses are not upskilling their staff to securely manage and deliver their IoT initiatives. If there is a shortage of skills in key stakeholder positions, there will almost certainly be an absence of awareness of the importance of cybersecurity.”

This conclusion was drawn from Inmarsat’s 2020 research report about IoT trends, titled The Rise of IoT in Mining, which found that respondents were aware of the obvious damage that a cyberattack could trigger and the range of threats they are now facing as they ramp up IoT adoption. In particular, the insecure storage of collected data (56%), employees misusing data (54%), insecure network links to external infrastructure (49%) and ransomware and malware (45%) were the most highly ranked security risks. Almost two-thirds (64%) of respondents reported lacking the level of security skills required when it comes to successfully delivering IoT-based solutions.

A less obvious, but potentially damaging impact — increasingly onerous bank loan terms — was recently highlighted by two authors writing in The American Accounting Association’s Accounting Review magazine. Henry He Huang of Yeshiva University and Chong Wang from The Hong Kong Polytechnic University studied the effect of data breaches on companies’ bank loan terms and found that those that had experienced breaches faced higher loan spreads and their loans were more likely to require collateral and demand more covenants. Although the article focused on more than a hundred public U.S. companies across a broad spectrum of business types and did not mention mining directly, the implications of perceived “information risk” for any company that uses traditional funding sources are obvious.

Tracking the Trends

Typically, two reliable indicators of whether certain themes might be just investor-friendly buzzwords — or genuine action items — are corporate hiring and contracting trends; companies retain the service providers and the people they need to accomplish critical goals. From this perspective, it appears that awareness is catching up to reality. Lately, mining companies, larger companies in particular, have been hiring Information Technology (IT) workers at an increasing rate. According to GlobalData’s monthly report, IT hiring in mining rose almost 13% from March to April 2021, with the top five most actively-hiring companies consistently accounting for slightly more than half the increase. Although IT jobs account for low, single-digit percentages of the global mining industry’s hiring activity in total, the hiring rate for IT workers has increased in 2021 concurrently with announcements from mining companies regarding AI and big data contracts, initiatives and applications. A few recent examples also illustrate how mineral producers are structuring and, in some cases, restructuring their digital transformation efforts through multi-vendor agreements and partnerships.

In June, for example, BHP announced it selected Amazon Web Services (AWS) and Microsoft as long-term cloud providers to support two distinct parts of its operations. BHP said AWS will provide capabilities in data analytics and machine learning to rapidly deploy digital solutions globally to improve operational performance; and Microsoft, through its Azure platform, will host BHP’s global applications portfolio. This, according to BHP, will enable the company to leverage its existing Microsoft licenses and SAP applications portfolio and reduce reliance on regional data centers.

BHP Chief Technical Officer Laura Tyler said, “Cloud is the foundation to our plans, and it will enable us to deploy digital solutions to the frontline quickly and at scale. Cloud will dramatically reduce the amount of hardware on sites and reduce costs.” As part of the effort, BHP will launch an AWS Cloud Academy Program to train and upskill their employees on AWS cloud skills.

Japanese conglomerate Itochu Corp. also recently announced it signed a memorandum of collaboration with the Brazilian CSN Group (Companhia Siderúrgica Nacional) for digital transformation (DX) to increase operational efficiency at its Casa de Pedra Mines iron ore operation in Brazil.

Itochu said it partnered with several vendors and will “…utilize the networks in the metal and mineral resource industry that it has built through previous interest investment and management as well as trade business to discover DX needs, introduce them to solution vendors, and take a leading role in DX matters, focusing primarily on mines where Itochu owns interests. It will also offer packages of multiple solutions to suit the needs of each mining company as well as make recommendations and provide initial support to those companies.”

GE Digital began a DX project at CdP in April, the first stage of which is to implement solutions to visualize operational data and administrative and financial data for managers as well as to conduct preventive maintenance at plants and relevant equipment and to analyze and improve the efficiency of operations. It is expected to contribute to improving business profitability mainly through cost reductions and improvement in metal recovery rate. In the second stage and beyond, the aim is for further efficiency improvement by linking data from mines to ports as well as more advanced data linking of Itochu functions and market information.

Along with GE Digital, other vendors participating in Itochu’s DX initiative include:

• Shell, which as part of its digital portfolio offers Oren, a B2B digital market- the place developed with IBM, which presents a number of digital and technology solutions for mining applications.

• MineRP, a South African company that, according to Itochu, provides an integration platform that consolidates data from unconnected expert mining technical systems into a digital twin of the entire mine and all assets.

• Sensyn Robotics, a Japanese company that develops robotics technology solutions.

• ScanX, a digital platform that rapidly handles data from laser scanners to assist end users by employing high-speed, optimized survey processing software.

Clean Data Takes Time

An interesting side note to ScanX’s cloud platform description is that, in the company’s words, it uses advanced classification algorithms and cloud-computing power to significantly reduce time-consuming work such as filtering useful information out of massive point cloud data, transforming raw data into actionable business insights.

A recent study that highlights how much time IT personnel actually spend getting data into usable form for making business decisions shows how useful automated “cleansing” of data can be. A 2020 survey by Anaconda, a software developer focused on innovative open-source data science applications, gathered responses from almost 2,400 data scientists, researchers, developers, analysts, data engineers, business managers, and others in various industries and from more than 100 countries showing that getting data science outputs into production, where they can impact a business, isn’t always straightforward. According to Anaconda’s summary: “Respondents reported that on average, 45% of their time is spent getting data ready (loading and cleansing) before they can use it to develop models and visualizations. Once their models are ready for production, they contend with numerous environments, dependencies, and even skills gaps before models see the light of day.”

The prospects, as illustrated by the Anaconda survey, of having highly paid IT workers and data scientists spending perhaps a third to half of their work time cleaning and preparing data, then encountering internal and external roadblocks before the clean data can be applied in a model, should be of concern to mining companies interested in optimizing productivity. But the challenges don’t end there; IT organizations might also have to contend with other issues such as:

• New sources of useful data that continue to proliferate at a rapid rate, bringing with them an expanding need for effective data management and control. Paul House, CEO of the mining-tech company IMDEX, recently noted that the take-up of new technologies is happening on a scale that has not been seen in the past, but every tool and technology that is added to a mining company’s arsenal — from exploration to production — increases the “attack surface” for hackers. His company, said House, has countered by achieving certification for ISO27001, an international information security standard recognized in 161 countries. According to House, clients have increasingly been asking for such security protocols to be in place.

• Zombie and orphan projects. Andrew Crose, vice president–Autonomous at Hexagon’s Mining division, wrote in a recent blog post that when different teams — operational staff vs. project staff, for example — pursue different goals or are otherwise unaligned, it often leads to zombie projects, which “never get handed over from the project team to the operations team. They linger, never quite alive, never quite dead, draining resources while the company pays a deployment team over scope. This also leads to orphans: the project team finally hands off to a begrudging operations team, who deny it the attention for maintenance and future improvements needed to deliver the intended value.”

In any of these cases, it’s likely that an IT team will be tasked with the responsibility for making available data from new technologies or projects accessible to all involved parties, in a useful format, a timely manner and security-standard compliant.

Cashing in on Digitalization

The stakes for effective data management are high. The World Economic Forum (WEF) predicts that digitalization in the mining/metals industries has the potential to deliver over $425 billion in shareholder, customer and environmental value by 2025. Much of this value will come in the form of operational effectiveness, increased productivity, faster decision-making, and enhanced safety. However, value won’t be achieved without some changes to the way mining organizations operate, and this won’t happen overnight, asserts the WEF.

“These industries need to consider an Integrated Operations Management (IOM) approach to gain holistic insight into their operations from design to maintenance,” said Craig Hudson, business development manager for mining metals and minerals for the Forum’s Anglophone region. “What does this mean in practice? Well, it means bringing all the data into a central location, using advanced analytics, and business information platforms to gain a real-time view of the complete value chain. Real-time contextualized data, driving business KPIs, is critical to unlocking the hidden value in the value chain of any operation.

“This is achieved by leveraging open and interoperable architectures as a platform to gather data in your operations. This increases your field-to-enterprise transparency and enhances value chain visibility and efficiency along the complete asset lifecycle, moving you from the mining business to the business of mining,” added Hudson.

It also opens a broad avenue for service providers and software systems vendors to develop solutions for managing data critical to enterprise success, ranging from exploration drillhole results to time-series data associated with physical asset performance, and beyond to sustainability and decarbonization outcomes.

With the recent trend of some mining companies to link corporate credit arrangements and loan structures to ESG performance, the availability of accurate and secure sustainability-related data may become an even more important factor in lending practices. In March, for example, Newmont became one of the first companies in the mining industry to add a sustainability-linked pricing component to its revolving credit facility. Under this arrangement, the company will incur positive or negative pricing adjustments on drawn balances based on certain sustainability performance criteria.

VOXEL, Anglo American’s DX platform, includes a sustainability module that, according to the company, has capabilities to do power analytics, methane gas analytics, environmental and biodiversity impact tracking, and greenhouse gas modelling.

The importance of ESG data isn’t being overlooked by consulting and professional services providers, either. For instance, enterprise mining software developer RPMGlobal recently announced that it planned to acquire Australia-based Nitro Solutions, which provides ESG services in the areas of environmental approvals, impact assessment, regulatory advice, environmental audits, compliance reporting (due diligence) and environmental economics, policy and legislation advice.

RPMGlobal Chief Executive Officer Richard Mathews said the acquisition combined with RPMGlobal’s existing ESG capabilities would allow the company to form a dedicated ESG division. “We also intend to harness the deep ESG domain knowledge of the Nitro team to assist RPMGlobal’s technology division identify software products we can either acquire, or alternatively build to service this accelerating market segment.”

Framing the Issues

As Seequent, developer of Leapfrog 3D geological modelling software for mining, points out in its latest Geoscience Data Management Report, geoscience data underpins every major decision made on subsurface projects. Their 2021 study, the most recent in a series that began in 2011, looked at some aspects that hadn’t been researched before, such as data management frameworks, ease of data access and data practices (e.g., data science scripting, advanced analytics, AI, Machine Learning). The latest results show that:

• Geoscience data management remains at least a top five issue for over 80% of organizations, yet less than a third currently use an established framework in their management processes.

• More than half of organizations are currently using or considering data practices such as data science scripting or machine learning/AI/advanced analytics.

• 35% of respondents stated they spend greater than 30% of their time on data management tasks. Over time, there has not been a significant reduction in time spent on data management tasks and only 1 in 10 claim it is extremely easy to access data within their organization.

• 40% of respondents stated version control was their biggest challenge when collaborating on active project data. Confidence in managing data quality and  quantity has remained consistent in the last five years which demonstrates there is still room for improvement.

• Since 2017, there has been a 12% increase in respondents wanting one point of access for data, and a single source of truth. The top two desired outcomes for data management challenges are getting full value from data and having one source of truth.

• 69% of respondents stated their readiness to leverage the cloud for geo-science data management. This is an increase of 14% since 2017.

It’s reasonable to assume that the data concerns and challenges examined in the Seequent study can be extrapolated to other areas of mineral extraction as well. Recent product and platform introductions and upgrades from a number of solution providers highlight some of the technological advances now available to mining organizations in the way of improved visualization, faster access and deeper insights gained from better data collection and analysis in almost every phase of mining.

For example, the accuracy, accessibility and analysis of mineralogical data gathered in the early stages of a mine project can have critical implications for mine planning, flowsheet design and production targets.  IMDEX recently announced that it has combined two data analysis products — ioGAS for geoscience analysis, and aiSIRIS for spectral mineralogy interpretation — to offer users a powerful solution to merge and analyze spectral mineralogy data with geochemical and other geological data sets.

IMDEX describes aiSIRIS is the first commercially operating artificial intelligence spectral mineralogy interpretation system and a leader in automated spectral mineralogy from handheld infrared spectrometers. Dr. Luisa Ashworth, IMDEX technical product support specialist, said that until aiSIRIS was developed, spectral mineralogy was confined to spectral experts using old style software, resulting in long turnaround times and often delivering incomplete and inaccurate results.

“aiSIRIS generates a standardised output which is the first of its kind in spectral mineralogy and has been trained on more than 2 million real world spectra, each of which has been interpreted in detail by a world class spectral expert, meaning it is robust across all common geological systems” Ashworth said. “aiSIRIS is producing expert-level interpretations much faster than a person would be able to with more accuracy than most spectral experts. It’s already clearly at the technical forefront of the industry and we are developing it to go further.”

ioGAS Product Manager Putra Sadikin said the integration created a powerful data analysis workflow that delivers detailed rock knowledge analysis addressing key “pain points.”

“The first pain point is resolved by the way aiSIRIS automates spectral data interpretation using a cloud-based solution. Once you push the automated mineralogy data to ioGAS, it addresses the second pain point which is how do we easily find patterns in that data and get better value out of the mineralogical information?”

Needed: Better Visualization

MineRP, one of the technological partners involved in Itochu’s DX initiative, recognizes that the challenges of mining start with the simple fact that most easy-to-reach deposits have already been tapped. The remaining orebodies are usually deeper, more remote or associated with difficult geological conditions. Developing a viable way to understand an orebody and its potential financial value, plan the mining and excavation, and execute the job requires generating and bringing together large amounts of data about the mineral resource being mined.

As an example, geological block models are one of the most critical pieces of data for making an informed decision about where to dig. “A geology block model with 30 million records in it would actually be on the smallish side,” noted Anton Van Der Walt, chief digital officer at MineRP. “It’s not unusual to have terabytes of data representing mine technical datasets.”

MineRP said its key innovation has been to integrate these masses of geological data with other datasets — from mine planning, design, and scheduling to ERP systems, IoT systems, and other systems required to run a modern mine ­— to give mining customers a single integrated platform for all their mining data. But in order to enable customers to bring together and spatially visualize, analyze, and even animate disconnected datasets within its MineRP Platform, the company needed a foundation of powerful visualization capabilities.

HOOPS Visualize — a high-performance 3D rendering engine from Tech Soft 3D — provided an ideal solution for MineRP, which claims that using HOOPS Visualize in the MineRP stack allows a customer to look at a mine, quickly identify the low-grade and high-grade areas, and then decide where they want to focus their extraction efforts.

Tech Soft 3D said its flagship graphics engine has powered cutting-edge graphics in hundreds of successful applications for more than 20 years, and its current partners in the mining and geoscientific markets include Geovia, Hexagon Mining, Halliburton and Trimble.

Real-time awareness of crucial changes in mine pit-slope or TSF stability is increasingly necessary to maintain a safe environment for mine workers and for communities adjacent to mining operations. Manual data collection of information needed for design and monitoring of these critical areas is gradually being replaced by increasingly sophisticated tools. Here are two examples:

Leveraging Data to Keep Pits and Dams Safe

Rocscience and IDS GeoRadar first joined forces in 2018 when deformation monitoring data from IDS’s monitoring radars was integrated into Rocscience’s 3D software, Slide3 and RS3. This integration, according to Rocscience, was the first of its kind in the industry and paved the way for radar data to support numerical modeling. In 2019, RS3 and Slide3 were given the ability to import live radar monitoring data, amplifying the scope for intelligent slope design.

More recently, the two companies worked together to bring Slide3 Safety Maps into Guardian, IDS GeoRadar’s software that manages slope stability risks in both geo-hazard assessment and open pit mining operations; and Rocscience’s Slide3 3D Limit Equilibrium Tool which allows geotechnical engineers to calculate the factor of safety of complex 3D slope stability geometries. Guardian enables fast interpretation and therefore fast reaction time to vital real-time information delivered by the IBIS and Hydra radar systems; and with Slide3, users can accurately calculate failures in any direction without the need for defining the direction in advance, increasing efficiency in modelling and analysis.

Users of Slide3 and Guardian can now export safety maps and contour maps from Slide3 in the form of a CSV file, and import those files into Guardian for further analysis. Users have the option to import one or more Factor of Safety maps then visualize in 3D over the Displacement/Safety map one of the imported Factor of Safety map to cross-compare the Slide3 simulation results with the radar measurement.

“Closing the loop between radar monitoring and slope stability modeling” was a significant step toward achieving a live monitoring-to-modeling approach, said Niccolò Coli, vice president, MineMonitoring at Hexagon Geosystems and managing director at IDS GeoRadar, which has been part of Hexagon since 2016.

“At Rocscience, our goal is to create more value for our users,” said Dr. Thamer Yacoub, CEO and president of Rocscience. “And with such product integrations we are enabling our users to bridge the gap between real data and modeling data, thus allowing them to achieve more reliable results.”

Renewed global concern about the risks of tailings dam failure is accelerating a revolution in the use of automated sensors to generate big, real-time data to better monitor and manage these facilities, according to SRK Consulting.

“The need for knowing more about tailings dam conditions — and in real time — has become a major focus within the mining sector, demanding a step change in the way we collect, process and interpret data,” said Lyzandra Boshoff, principal engineering geologist at SRK Consulting. “As part of these efforts, SRK has been rolling out initiatives using automated vibrating wire piezometers (VWPs) on tailings facilities.”

Boshoff highlighted the particular significance of seepage and the associated pore pressure regime within a tailings facility as a vital aspect of the integrity and stability of the structure. This has conventionally been tracked by manual standpipe piezometers whose performance, while accurate, depends on the quality of installation and aftercare – and manual data collection is subject to human error.

“This means expending considerable effort for relatively little data, which may often not exactly reflect the current situation by the time the information reaches the engineer for analysis,” she said. “Even the automated sensors using vibrating wire technology tended to rely on manual data collection from the logging devices connected to the sensors.”

This is now changing as many telemetry hardware developers have improved the wireless capability and cloud technology to upload and store data, allowing data to be immediately accessed by engineers and management.

“This has changed the landscape of data flow and interpretation, as well as the potential size of the datasets that can be generated,” she said. “SRK has been at the forefront of rolling out automated VWP networks, using logging systems that can send data wirelessly to cloud-based databases and then visualized and analyzed in real time.”

Among the technological innovations the company is developing is a customized database and visualization platform specifically for VWP data. This includes built-in, automated validation and interpretive tools to automate some of the routine engineering interpretive work. It builds upon available software technologies to create a solution that is customized to the field of tailings dam management, while also presenting opportunities for applying the principles in a range of disciplines.

The datasets generated by VWP networks can be significantly large, depending on the frequency at which data is collected. Conditions on site would determine the detail required for tracking where potentially adverse circumstances may develop.

“These VWP networks are also useful when needing to confirm whether the mitigation measures in place are having the desired effect,” said Boshoff. Applying these technologies, SRK has been able to review the impact of construction process and other site activities and incidents on pore pressure responses and analyze the impact of depositional patterns on the seepage flow regime within a facility.

“For the first time, we can see and correlate in real time what we have always predicted using models and assumptions,” she said. “Harnessing the power of big data, we can now test our assumptions and substantially raise the confidence of our observations. With the exponential growth in the application of technology in this field, more data is being generated and is available to be harnessed and interpreted.”

The sheer quantity of data generated means that engineers must innovate, finding new and more efficient ways to validate, evaluate and interpret the large incoming datasets. She notes, however, that harnessing the full power of big datasets demands the appropriate database structures and validation processes to be in place, which is where SRK is investing considerable effort.

“To support this need, we have recently put in place a data services department at SRK, pulling together some of our brightest minds and data analysts,” she said. “This is helping champion our initiatives to harness these growing datasets into sound engineering information — so they can contribute to making informed engineering decisions.”

The Importance of Data Literacy

In a data-driven industry or company, a higher degree of data literacy — the ability to understand, work with and apply data solutions for maximum business value — throughout an enterprise will be crucial to meet competitive pressures. This is a given for IT workers and also presents a challenge to the mining workforce in general, which faces some major training and skills-related shifts in the coming years (see sidebar Big Changes Ahead for the Mining Workforce), but corporate executives won’t be exempt, either. Although a certain amount of data literacy has existed at C-suite levels for many years, the rising influence of data on business practices means that erroneous data-based high-level interpretations or decisions have the potential to cause more damage than ever before.

What does it take to become data literate? A paper published several years ago (Approaches to Building Big Data Literacy, Catherine D’Ignazio, Emerson Engagement Lab; and Rahul Bhargava, MIT Center for Civic Media, Bloomberg Data for Good Exchange Conference, 2015) offered guidance; specifically, four major points: the ability to read, work with, analyze and argue with data.

“Reading data involves understanding what data is, and what aspects of the world it represents. Working with data involves creating, acquiring, cleaning, and managing it. Analyzing data involves filtering, sorting, aggregating, comparing, and performing other such analytic operations on it. Arguing with data involves using data to support a larger narrative intended to communicate some message to a particular audience,” the authors wrote.

A recent blog post from Anaconda, the software development company mentioned earlier in this article, also underlines the importance of data skills for business leaders, noting that there’s often a gap between what executives want to see and what data scientists want to do. Business leaders expect data scientists to generate a clear-cut recommendation on where to steer the company, while data scientists seek room for exploration to test out hypotheses about what their data shows. “To bridge the gap, both sides have a responsibility to understand and speak the other’s lingo. When reviewing the results from a given model, decision-makers should be able to ask questions like: “How did we select the data for this model?” “How heavily do different parameters influence the model’s output?” “How strong are the confidence intervals?” and “How can we test for bias?”

“A thorough understanding of data begins with accepting the fluidity of data —recognizing that the straight-line graph that we so often desire is usually an oversimplification. Working with real data may produce shaded regions or error bounds that may not paint a perfectly clear picture but are nevertheless highly valuable in guiding leaders toward a decision,” the blog concluded. “Just as the most successful data experts learn how to communicate their work to non-technical audiences, so too do the best business leaders learn how to be more informed partners to their data teams.”

---

Big Changes Ahead for the Mining Workforce

The World Economic Forum, in cooperation with the International Council on Mining and Metals (ICMM), published a sector-specific report in 2021 that summarized key findings from the WEF’s The Future of Jobs Report 2020. These findings include a number of mining workforce insights, some of which are listed below:

• The mining and metals companies surveyed were primarily focused on the adoption of three key technologies in the next four years: non-humanoid robotics; internet of things (IoT) and connected devices; and big data analytics. But 73% of companies identified skills gaps in the local labor market as the biggest barrier to the adoption of such new technology.

• Companies indicated that an average of 48% of existing employees would require reskilling/upskilling in the next four years to meet the evolving skills required for the tasks they would have to perform.

• In the next four years the average employee’s skillset will change by 40%. Most companies expected existing employees to pick up skills on the job and assumed this would be their primary strategy to address the shifting skills demand.

• Ultimately, the average share of mining and metals workers at risk  of displacement was estimated at 20% (compared with 14.2% in oil and gas, and 11% in agriculture, food and beverage).

• Despite differences in skills and work profiles, the types of emerging skills and jobs, the types of redundant jobs and the focus of current reskilling and upskilling programs were similar across different global regions.