File processing systems were the common way digital data was stored before databases and database management systems were developed. As the name suggests, FPS systems stored data in disparate files which were accessed by applications.
FPS systems are supported by underlying systems software, such as operating systems, and subject to the file-based constraints of those systems. In some cases, this dependency causes disadvantages such as adequate permissions management. The FPS system has many advantages and disadvantages to consider.
Anatomy of File Processing System
A file processing system may use files such as .txt, .jpg, .docx, or even structured datatypes such as .xml or .json. Files may be placed in different system folders for organization or more complex management requirements.
In most ways, FPS systems resemble the conceptual framework provided by common operating systems like OsX, Windows, and Linux. Files, directories, and metadata are all accessible and able to be manipulated. Additionally, a system-supported file processing system will have space-management services available (R).
Advantages
File processing systems were the first approach at managing data after the advent of computers and the term was used as early as 1961 (R). It was a logical progression from temporary data states, such as those held in Random Access Memory (RAM) and allowed users and developers to preserve information between computing sessions. Particularly, FPS systems offered these advantages:
- Budget Friendly – In most cases, no third-party tools beyond the operating system is required for file processing systems.
- Easily Scalable – File sizes and storage space can easily be extended, though some aspects of scalability can be limited.
- User Friendly – File processing systems mimic those familiar to operating system structures such that users are often able to seamlessly interact with data.
Disadvantages
The advantages of file processing systems are, largely, recognized when compared to very outdated or very limited systems. For example, FPS systems really shine when comparing their ability to persist data compared to that of RAM. Below are common disadvantages of FPS systems in the context of most modern use-cases:
Data Redundancy
FPS systems store data in application-specific ways such that disparate applications do not commonly share common data. However, data may sometimes represent common entities, either wholly or partially. For example, a University’s CS department and Math departments both maintain grading records for students.
The grades from either department are essential to calculating total GPA. However, when one department updates the grades of one student’s records those grades aren’t updated on the other department’s records for that student. As such, a very different GPA calculation could easily result.
Data Access Issues
FPS systems rely on developer foresight to dictate the ability to filter and present data. This restricts the flexibility with which users are able to access specific data. For example, an FPS system storing pricing data for NASDAQ-listed securities wouldn’t be able to filter based on certain attributes like price–unless that function was explicitly defined by the developers.
A user, intending to find stocks with current prices in a given range, would have to query all records and then manually sort them. In practice, this would be equivalent to downloading a pricing sheet from NASDAQ, opening that file in a program like Microsoft Excel, and filtering the data there.
Data Isolation
FPS systems store data on an application level–each application has its own copy and versioning of data. When creating new applications, this makes defining how existing data may be accessed, where it may be accessed from, and how local copies may be duplicated and/or stored.
With a database, all information is centralized and access controls are global for any accessing application. This allows developers to maintain a central set of data access procedures whereas an FPS system requires a unique definition for every application.
Data Integrity
Data often requires certain constraints be enforced to ensure accuracy and usability. Examples include passwords requiring special characters, usernames being unique, or floating point values being stored with a certain precision. Databases are able to enforce such policies centrally such that all accessing applications are subject to their specifications.
File processing systems must enforce such restrictions at the application level. This means developers must maintain a cadence to mirror changes or extensions to data restrictions otherwise one application could invalidate the data used by another. For example, truncating a 6-place-precision value to 2 places.
Atomicity
Atomicity refers to the most fundamental constituent part of a larger whole, such that divisibility isn’t possible. In database systems, this describes situations where multi-stage data access must have all stages complete successfully otherwise all must be negated. Upon such negation, the initial state of data must be restored. For example: someone transfers $500 from one their checking account to their savings account.
In this case, the atomicity is contextual to the $500 such that it should be reflected in only the checking account if either the transfer from the checking account failed or the transfer to the savings account failed. In other words, if the money doesn’t get deposited to the savings account it shouldn’t be debited from the checking account–that’d be robbery!
Concurrency
Access to data from multiple applications and users is common in modern computing. File-based systems aren’t able to effectively manage such requests. For example, a major online retailer would need to facilitate multiple purchases simultaneously–resulting in decrements of inventory, storage of order details, and debiting of customer accounts.
File-based access systems would prohibit the simultaneous access as such, and would rely on application-specific implementations of threads, queues, and redundancy checks. While conceivable on smaller scales, this approach quickly becomes too complex to manage effectively within the domain of many common enterprise cases.
Security/Permissions
File-based systems rely on the administration of underlying systems architecture to enact security. This greatly limits the ability of applications to enforce granularity in user-access to data. For example, as common operating system security might enact: an application would effectively be assigned a user-based permission for file and directory access.
However, that application would be unable to enact granularity in its users access to data without explicit developer definition. Once again, this would be feasible on smaller scales but not within more complex use cases common to modern computing. At the very least, it would present a hurdle for developers of multiple applications to develop a unified security protocol.
Final Thoughts
File Processing Systems were, in the early days of their use, heralded as major advances in data management. As applications in computing have grown in complexity however, they have quickly become less-than-ideal solutions.
Database management systems serve as their replacement in most modern applications but there are still certainly use-cases where an FPS may be useful–most notably in personal computing, rapid application development, and some disparate cases of data analysis.
