Open data transfer
The real and most important motive for publishing data in a free mode is the need to find new solutions, new methods, models and technologies - new knowledge in the designated subject area.
Free and open dissemination of data (open publication) is a special case of a system of transactional target information exchange between two subjects in which the first transmits information to the second in accordance with an agreement (contract).
Having considered the motivation of open data, we will try to understand the essence of the process of transferring data to an unlimited or conditionally limited circle of persons and highlight its distinctive features in comparison with a closed scheme.
')
In today's “computerized” economy, without the exchange of information, and in particular digital data, it is almost impossible to maintain management at a high level.
Moreover, the role of intensive data exchange is constantly increasing due to a number of factors:
Given the active pressure of these factors, economic entities, within the framework of their own data needs, are forced to solve several important issues for themselves:
- where to get the data? - selection of data sources;
- what data is needed? - evaluation and filtering of the composition (content) of the data;
- how to manage the data? - the use of equipment and tools;
- what to do with the data? - the choice and use of methodologies and technologies;
- what is the advantage of data? - search for a better solution based on data;
- what are the risks associated with the data? - optimization of work with data on costs and effectiveness (consequences).
And of course, answering such questions inevitably raises the technological problem of exchanging data with a selected circle of counterparties, government organizations, consumers, suppliers, competitors, experts, communities. The solution to this problem is the data transfer mechanisms between two or more subjects that are implemented within a variety of platforms, systems, protocols, etc.
We will agree to use the term “data transfer” to transfer data from one subject to another using such mechanisms in accordance with a given contract.
Data transfer - data transfer by one entity (supplier) to another entity (recipient) in accordance with agreed terms (contract) for joint or parallel realization of economic, marketing and management goals (tasks).
Three key tasks facing information management have a direct impact on the development and improvement of the efficiency of tools, technologies, quality and volumes of data transfer:
But each of these tasks is solved in its own way based on the type and purpose of the data used, taking into account the roles played by the subject in the transfer process.
When considering the data transfer scheme or even going a little beyond its scope, it is worth distinguishing nine roles of the subjects of the participants :
In the simple case, the first three roles (1-3) are implemented by one subject, and the next three roles (4-6) are implemented by another subject.
If the recipient of the data passes the identification, authentication, subsequent authorization on the operation of obtaining a specific data set and is restricted in using the data set, then it is a closed or protected data transfer scheme .
This means that the supplier directly transmits to the recipient from a limited, identified and trusted circle of persons a final and known set of data in accordance with a bilateral agreement (contract) on the quantity and quality of data and the method of sending and receiving them.
If the data recipient does not pass the identification or passes the formal identification and does not pass the authorization during the operation of obtaining a specific data set and is not limited to the subsequent use of the data set, then it is a public (public) data transfer scheme .
This means that the supplier directly or indirectly transmits to the recipient from an unlimited or formally limited circle, not identifiable or formally identifiable circle of persons a given set of data in accordance with a public agreement (contract) on the quantity, quality and method of their transfer, to which the recipient joins.
Several key points for comparing the two data transfer schemes:
A secure data transfer scheme is used, for example:
At the same time, the protected-scheme has the following features:
In contrast to the secure, public data transfer scheme applies including:
Such an application explains the features of the public scheme:
Each of the options for the data transfer scheme is useful for a particular task and the specific choice depends on the goal.
The main criterion for choosing between protected and public schemes is of course the value of the data itself. If this is data that determines the competitive advantage of a business (organization) or is its trade secret at all, then a public scheme cannot be considered as acceptable, even if the distribution of data is intended among a wide range of persons (partners).
It is necessary to take into account and understand that the data transmitted publicly is:
Data in open access for a large number of consumers must have a certain degree of “independence” and “independence”.
This is due to the fact that the data provider will not be able to explain to each user their purpose, composition, quality characteristics, etc. By the way, here comes the question of accompanying public data with the same public metadata.
Once again…
The public data transfer scheme is a bad scheme for transferring data constituting commercial or state secrets, as well as special data, the distribution of which is limited by the current legislation.
Let us leave aside the topic of secure data exchange, especially since a large and important part of them comes down to tools and technologies for protecting communication channels, algorithms for reliable encryption during transmission, protection of integrity, etc.
Openly published data does not necessarily have no restrictions on use.
Even more likely the opposite. Public data necessarily have a number of restrictions, including legal clauses. The simplest of these is usually available directly or via the link at the bottom of the page for downloading an open set of digital data and it goes something like this: “the data user must indicate the link to the source of the open data”. This is just one of the mandatory legal aspects related to the published data. Something is regulated at the legislative level, and something at the level of corporate regulations.
But with respect to free data, legal regulation rather plays a permissive-explanatory role. Strict licensing and strict copyright does not make much sense to apply to digital data published for a wide range of consumers. What then is the point of transmitting data openly, if you still cannot fully control the fulfillment of the conditions of the issued licenses or copyright agreements?
Data on a free transfer must always be truly free to be distributed.
In any case, the supplier can never completely control the number of copies made by the consumer.
The potential to somehow limit the use of data transmitted publicly exists - this is a declaration of restrictions. And you can declare restrictions on both data access and data usage.
Restriction of access to data (options):
Restriction of use of data (options):
But all these restrictions are conditional and are only declared - formalized and stated - by the data provider. A number of technological tools, of course, allows us to establish control over the execution of specified constraints, but only within some real limits. Significant restrictions will simply translate data from the category of publicly available to protected.
By the way, the purpose of the obligatory reference to the source of openly published data that suppliers have to demand is to establish a direct connection between public data and the solution created on the basis of it (knowledge, product, service, etc.). Reference to the source of digital data provides an important feedback from analysts and users. It is the easiest way to control the use cases of free data.
If we return to the definitions of open data that were given in the previous publication , then they lack an indication of the subject who discloses them. It is assumed that anyone can freely distribute data. Most likely for social, cultural or political significance there is really no difference who published the data: state, business or citizen. But from the point of view of economic relations and competing markets - this is important.
PS This publication continues the thematic series on open data, begun in the article Why data can be open and free and is solely the author's point of view not claiming to be the ultimate truth.
Free and open dissemination of data (open publication) is a special case of a system of transactional target information exchange between two subjects in which the first transmits information to the second in accordance with an agreement (contract).
Having considered the motivation of open data, we will try to understand the essence of the process of transferring data to an unlimited or conditionally limited circle of persons and highlight its distinctive features in comparison with a closed scheme.
')
In today's “computerized” economy, without the exchange of information, and in particular digital data, it is almost impossible to maintain management at a high level.
Moreover, the role of intensive data exchange is constantly increasing due to a number of factors:
- Digitalization of the economy : the data cover more and more aspects of human economic and social activities.
- Big data : the number and density of data is growing steadily and steadily.
- Everything is connected to the global communications network : in a single information network there are a lot of generators and consumers of various digital structured data.
- Information mobility : information (and digital data in particular) is quickly created and distributed both by copying and by reference, and becomes available in any geographic location where there is Internet access.
- Flexible communities : work with data is organized in various ways and levels within the framework of formal and informal communities that are actively forming and transforming into networks, and those in turn create and redistribute new primary and secondary data and distribute them.
- Open interaction : focus on the involvement of both individual citizens and professional communities or businesses in the formation of an effective socially just and guaranteeing the security of the system of regulation and control of joint activities of citizens and their social groups.
Given the active pressure of these factors, economic entities, within the framework of their own data needs, are forced to solve several important issues for themselves:
- where to get the data? - selection of data sources;
- what data is needed? - evaluation and filtering of the composition (content) of the data;
- how to manage the data? - the use of equipment and tools;
- what to do with the data? - the choice and use of methodologies and technologies;
- what is the advantage of data? - search for a better solution based on data;
- what are the risks associated with the data? - optimization of work with data on costs and effectiveness (consequences).
And of course, answering such questions inevitably raises the technological problem of exchanging data with a selected circle of counterparties, government organizations, consumers, suppliers, competitors, experts, communities. The solution to this problem is the data transfer mechanisms between two or more subjects that are implemented within a variety of platforms, systems, protocols, etc.
We will agree to use the term “data transfer” to transfer data from one subject to another using such mechanisms in accordance with a given contract.
Data transfer - data transfer by one entity (supplier) to another entity (recipient) in accordance with agreed terms (contract) for joint or parallel realization of economic, marketing and management goals (tasks).
Some basic data transfer examples
Business regularly sends data to state tax authorities for calculating and checking tax payments in accordance with the law - this is a transfer of data from an economic entity to the state.
The consumer transfers his passport data for processing a loan agreement with the bank according to the norms of the law on commercial banks and according to the rules of the credit institution itself - this is a transfer of data from a citizen to a bank.
The State Statistics Service publishes data on unemployment on its own website on the Internet - this is a transfer of data from the state to business and citizens who are in search of a new job.
The consumer transfers his passport data for processing a loan agreement with the bank according to the norms of the law on commercial banks and according to the rules of the credit institution itself - this is a transfer of data from a citizen to a bank.
The State Statistics Service publishes data on unemployment on its own website on the Internet - this is a transfer of data from the state to business and citizens who are in search of a new job.
Three key tasks facing information management have a direct impact on the development and improvement of the efficiency of tools, technologies, quality and volumes of data transfer:
- The task of increasing the amount of data used in the management.
- The task of development and complication of goals solved on the basis of data.
- The task of improving data management.
But each of these tasks is solved in its own way based on the type and purpose of the data used, taking into account the roles played by the subject in the transfer process.
When considering the data transfer scheme or even going a little beyond its scope, it is worth distinguishing nine roles of the subjects of the participants :
- data generator (author) - a subject who directly creates data;
- data owner - an entity that has recognized ownership of the data;
- data provider (publisher) - the subject who gives the data;
- data recipient - the subject who receives the data;
- data consumer - a subject who is interested in receiving data;
- data processor - a subject that receives and processes data;
- data controller - the subject who checks the data during transmission;
- data transfer controller - a subject that determines the rules and procedures for data transfer;
- data transfer coordinator - a subject who helps to transfer data and / or partially controls data transfer procedures.
In the simple case, the first three roles (1-3) are implemented by one subject, and the next three roles (4-6) are implemented by another subject.
If the recipient of the data passes the identification, authentication, subsequent authorization on the operation of obtaining a specific data set and is restricted in using the data set, then it is a closed or protected data transfer scheme .
This means that the supplier directly transmits to the recipient from a limited, identified and trusted circle of persons a final and known set of data in accordance with a bilateral agreement (contract) on the quantity and quality of data and the method of sending and receiving them.
If the data recipient does not pass the identification or passes the formal identification and does not pass the authorization during the operation of obtaining a specific data set and is not limited to the subsequent use of the data set, then it is a public (public) data transfer scheme .
This means that the supplier directly or indirectly transmits to the recipient from an unlimited or formally limited circle, not identifiable or formally identifiable circle of persons a given set of data in accordance with a public agreement (contract) on the quantity, quality and method of their transfer, to which the recipient joins.
Several key points for comparing the two data transfer schemes:
Protected circuit
- The recipient is identified and authenticated.
- Data access is carried out after authorization of the data recipient
- The format and content of the data are prepared in a special way.
- Terms of data transfer agreed by the parties in a closed form (document)
- Special data protection and transmission channels are applied.
Public scheme
- Recipient is anonymous or formally identified.
- Data access does not require authorization or requires formal authorization
- Format and data content are prepared in a standard way.
- Transmission conditions are declared by the data provider.
- Data and transmission channel protection is not applicable or is minimal.
A secure data transfer scheme is used, for example:
- for data exchange with economic partners (consumers, dealers, distributors, suppliers, outsourcing companies);
- to transfer credentials to government regulatory and regulatory authorities;
- for sectoral commercially significant information partnerships (self-regulatory organizations, associations, partnerships, consortia);
- as an intra-entity (corporate) transfer of commercial data: interdivisional, cross-border, etc.
At the same time, the protected-scheme has the following features:
- Closed schemes and exchange protocols.
- Enhanced security procedures.
- High requirements for reliability of information.
- Unchangeable format and composition of data.
- Agreed or regulatory data sharing (bilateral contract).
- A single or high-level information system provider and recipient of data.
- High risks in case of “leakage” of data in the exchange process.
- Special commercial (economic) value of data.
In contrast to the secure, public data transfer scheme applies including:
- for data exchange with secondary (opportunistic) economic entities;
- to transmit data for public review and control;
- for open information interaction with industry and non-core associations and experts (special events);
- as a corporate open social network (eg, sharing data with colleagues and active clients).
Such an application explains the features of the public scheme:
- Open procedures and exchange protocols.
- Simple security procedures.
- Confidentiality of information.
- Permanent control of the format and composition of data by the recipient.
- Declared terms of data exchange (contract of accession).
- Hypothetical incompatibility of information systems of the supplier and the recipient of data.
- Reduction of risks in case of “leakage” of data due to their preliminary filtering and aggregation.
- Data has weak or uncertain commercial value.
Each of the options for the data transfer scheme is useful for a particular task and the specific choice depends on the goal.
The main criterion for choosing between protected and public schemes is of course the value of the data itself. If this is data that determines the competitive advantage of a business (organization) or is its trade secret at all, then a public scheme cannot be considered as acceptable, even if the distribution of data is intended among a wide range of persons (partners).
It is necessary to take into account and understand that the data transmitted publicly is:
- data, the publication of which is unknown or it is not completely clear how and for what purposes the recipient will use them;
- data that is to be distributed not among specific individuals, but among conditionally limited or not limited at all set of users;
- data for which no control is made of the number of copies made;
- machine readable valid data;
- data suitable for processing (analysis) and further use (ie, representing some meaningful set of information on a specific subject);
- data that is explicitly or implicitly accompanied by metadata that defines: a scheme or data model, an assessment of the quality of the data, a purpose;
- data referring to the context (or giving such a reference);
- person-readable data (sometimes).
Data in open access for a large number of consumers must have a certain degree of “independence” and “independence”.
This is due to the fact that the data provider will not be able to explain to each user their purpose, composition, quality characteristics, etc. By the way, here comes the question of accompanying public data with the same public metadata.
Once again…
The public data transfer scheme is a bad scheme for transferring data constituting commercial or state secrets, as well as special data, the distribution of which is limited by the current legislation.
The public data transfer scheme is the way to publicly “open” data for a large number of unidentifiable users.
Let us leave aside the topic of secure data exchange, especially since a large and important part of them comes down to tools and technologies for protecting communication channels, algorithms for reliable encryption during transmission, protection of integrity, etc.
Openly published data does not necessarily have no restrictions on use.
Even more likely the opposite. Public data necessarily have a number of restrictions, including legal clauses. The simplest of these is usually available directly or via the link at the bottom of the page for downloading an open set of digital data and it goes something like this: “the data user must indicate the link to the source of the open data”. This is just one of the mandatory legal aspects related to the published data. Something is regulated at the legislative level, and something at the level of corporate regulations.
But with respect to free data, legal regulation rather plays a permissive-explanatory role. Strict licensing and strict copyright does not make much sense to apply to digital data published for a wide range of consumers. What then is the point of transmitting data openly, if you still cannot fully control the fulfillment of the conditions of the issued licenses or copyright agreements?
Data on a free transfer must always be truly free to be distributed.
In any case, the supplier can never completely control the number of copies made by the consumer.
Is it necessary for somehow for openly published data to refer to free licenses such as Creative Commons Attribution?
The question is not unambiguous. Each publisher has the right to respond to it based on their own legal position. But, again, do not forget about the main motivation of gratuitous disclosure of data. The data provider publishes them freely, not in order to then persecute users someday and somehow. If the data is opened, then it is allowed to do almost anything with it. And no persuasion in the form of sophisticated legal exquisite documents will not prevent the recipient of the data from processing it in some way “in its own way”, analyzing and transferring it to a third party. It is easiest for the publisher to explain in a clear form how he sees the use of the data published by him and what he expects from bona fide users.
The potential to somehow limit the use of data transmitted publicly exists - this is a declaration of restrictions. And you can declare restrictions on both data access and data usage.
Restriction of access to data (options):
- "Favorite" - access is obtained only by some individuals who are members of the target (industry) expert communities, associations, organizations, etc. (but at the same time they are not limited in the secondary transmission of data according to the public scheme);
- “Correct” - access is obtained only by persons who agree to fulfill the conditions set by the supplier and / or comply with the qualifying requirements of the supplier;
- “All” - any interested users have access, in fact, the absence of any restriction.
Restriction of use of data (options):
- “As agreed” - data can be used only on the established conditions of suppliers in a separate document;
- “As free” - data can be used freely, and a fee should not be charged for a copy of the data;
- “As you like” - data can be used without any restrictions, including even their copies can be transferred for a fee.
But all these restrictions are conditional and are only declared - formalized and stated - by the data provider. A number of technological tools, of course, allows us to establish control over the execution of specified constraints, but only within some real limits. Significant restrictions will simply translate data from the category of publicly available to protected.
By the way, the purpose of the obligatory reference to the source of openly published data that suppliers have to demand is to establish a direct connection between public data and the solution created on the basis of it (knowledge, product, service, etc.). Reference to the source of digital data provides an important feedback from analysts and users. It is the easiest way to control the use cases of free data.
If we return to the definitions of open data that were given in the previous publication , then they lack an indication of the subject who discloses them. It is assumed that anyone can freely distribute data. Most likely for social, cultural or political significance there is really no difference who published the data: state, business or citizen. But from the point of view of economic relations and competing markets - this is important.
PS This publication continues the thematic series on open data, begun in the article Why data can be open and free and is solely the author's point of view not claiming to be the ultimate truth.
Source: https://habr.com/ru/post/320420/
All Articles