How do organizations manage data, data storage and data bases?
Written by Rabih Soueidi, Managing Partner of Sapience Consultancy
When it comes to organizations, data is a critical asset that must be dealt with. Storing and managing data in time is becoming a necessity to make data-driven decisions. Most organizations have data governance plans which handle the overall management of data. Successful data governance strategies require careful planning, the right people and appropriate tools and technologies (TechTarget, 2017). These plans include procedures to store and replicate data in different layers for high availability. Also, plans for disaster recovery that may include tapes, cartridges, and large capacity USB drives with integrated data backup software (Ready, 2017), keeping tapes of backup in various locations; in safes and banks sometimes. Security is also a big concern, as unwanted access to data can jeopardize the whole business in some cases, and may endanger the lives of people. Depending on the line of business, data governance is planned, for example, a Bank would invest more money in securing its data than a car rental organization. But the main concerns are almost the same: availability, integrity, security, and usability.
Systems and Storage under IT departments handle the data centers of an organization and the apps running them. Usually, a data center has bare-metal servers along with storage technologies such as Storage Area Networks (SAN), Network Access Storage (NAS) and Directly Attached Storage (DAS) (Peng, 2012). The purpose of a data center is to provide systems needed to conduct business. These servers may be physical or virtual (or combination of both). Organizations have virtualization software deployed such as VMware and Hyper-V that allow them to access and create servers virtually and also help in backing up and restoring these servers. According to inspection done by Spiceworks, large enterprises heavily favor VMware while smaller organizations use Hyper-V at a higher rate (Tsai, 2016). Servers are gathered in clusters that are usually managed in a centralized way depending on the virtualization software. These servers run various apps needed by the company, but they also run apps that are critical in managing and governing users in an organization.
Databases can be SQL or NoSQL; they can be hosted on-premise or in the cloud. Historically, SQL databases i.e. relational database management systems were being used extensively in organizations because they are designed to store data for a line of business apps that need to store transactional data in an easy and consistent way. However, relational databases become inefficient in handling Big Data; first, they don’t scale well to very large sizes, second, they don’t handle unstructured data very well (Reeve, 2017). Tech companies started later creating their own database stores as new business models emerged. These databases are designed to be more scalable and available; they use clusters of sometimes commodity computers to scale out and handle more requests.
Many organizations these days are migrating parts of their infrastructure to the cloud. In Sapience Consultancy, all our research data is being stored and managed on Microsoft Azure Storage. We use Office 365 for our emails and file storage. We host our data visualization tools (front-end and back-end) on Azure Websites.
Ping, Yang (2012) ‘A Network Storage Framework Based on SAN’, 7th International Conference on Computer Science & Education, pp. 818 – 82.
Ready (2017) IT Disaster Recovery. Available at: https://www.ready.gov/business/implementation/IT. (Accessed: 8 December 2017).
Reeve (2017), ‘Big Data and NoSQL: The Problem with Relational Databases’, DELLEMC, 7 September. Available at: https://infocus.emc.com/april_reeve/big-data-and-nosql-the-problem-with-relational-databases/. (Accessed: 8 December 2018).
TechTarget (2017), ‘Building an effective data governance’, TechTarget. Available at http://searchdatamanagement.techtarget.com/essentialguide/Building-an-effective-data-governance-framework. (Accessed: 8 December 2017).
Tsai (2016), ‘Server Virtualization and OS Trends’, Spiceworks, 30 August. Available at https://community.spiceworks.com/networking/articles/2462-server-virtualization-and-os-trends. (Accessed: 8 December 2018).