Which of the following would be the best data source to identify opportunities to improve timeliness?

5.3.5 Timeliness

Timeliness refers to the time expectation for accessibility and availability of information. Timeliness can be measured as the time between when information is expected and when it is readily available for use. In the MDM environment, this concept is of particular interest, because synchronization of data updates to application data with the centralized resource supports the concept of the common, shared, unique representation. The success of business applications relying on master data depends on consistent and timely information. Therefore, service levels specifying how quickly the data must be propagated through the centralized repository should be defined so that compliance with those timeliness constraints can be measured.

3.10.2.2 The Features of Historical Data

As the foundation of HGIS, historical data have the following features:

Timeliness. Time attribute is the core attribute of historical data, and it is also quite different from modern data. Historical data should have clear chronological information in the first instance. In the cross-sectional study at a certain point of time, attention should be paid to the identification and unification of the time standard of each database. In the study of transition changes, focus should be on the selection and integration of data in order to form effective time series. In addition to transition changes, Man (2002) has also introduced the concept of growing period of geographic features. By extracting the beginning and ending of a certain state of a research object, Man then enables himself to describe the changes of the research object in the two dimensions of time and space. This method has now been applied to the construction process of CHGIS and other related research.

Spatiality. The occurrence of historical events and phenomena can be associated with a specific location. Spatial attributes are not directly linked with historical data, but are the necessary information base for HGIS research and application. Spatial information of historical data, on one hand, is derived from the original record of the document or excavation site; on the other hand, it also forms the derivative identification in the study. Man introduced the concept of carrier data, pointing out that “carrier data can be defined as the data which can carry the elements of other data and the implementation of the relevant spatial location of the data.” It is also recommended that it is necessary to select the appropriate carrier data to build up the spatial framework for historical data. The selection of carrier data usually requires the consideration of spatial resolution, the stability and possibility of the data in the time series and other related characteristics (Man, 2008).

Multisource. In addition to timeliness and spatiality, the thematic nature of historical data is another important attribute representing the main object and content of data description. Meanwhile, the source of historical data for a given topic is also diverse. For example, the number of schools recorded during the Republic of China period in Peking City may include the official historical records and investigations in the schools, official materials and archives on campus, as well as a travel guide for Peking City, celebrity memoires, and other forms. Among them, official investigations can be segmented into national, regional, and district levels. The diversity of data sources may, on one hand, offer good support for the comprehensiveness and continuity of the sample data; and on the other hand, increase the difficulty to ensure the accuracy and standardization of the data in the study.

8.2.3.1 Timeliness and Instantaneity

Ensuring the timeliness of processing requires the ability to collect, transfer, process, and present the stream data in real-time. As the value of data may vanish over time rather rapidly, the streaming architecture needs to perform all the calculation and communication on the fly with the data that has newly arrived.

On the other hand, the data generation in IoT environments mainly depends on the status of data sources. The amount of data that is generated at low-activity periods can be dramatically less than the number observed at peak times. Usually the stream-processing platform has no control over the volume and complexity of the incoming data stream. Therefore, it is necessary to build an adaptive platform that can elastically scale with respect to fluctuating processing demands, and still remain portable and configurable in order to stay agile in response to the continuously shifting processing needs.

11.2 Aspects of Availability and Their Implications

What do timeliness, currency, latency, consistency, coherence, and determinism mean in the context of synchronized master data? In Chapter 5 we explored the dimensions of data quality, yet the rules and thresholds for those dimensions are defined from the point of view of each of the business applications using the data sets. But as characteristics of a synchronized master view, these terms take on more operational meanings. Within each application, accesses to each underlying managed data set are coordinated through the internal system services that appropriately order and serialize data reads and writes. In this manner, consistency is maintained algorithmically, and the end users are shielded from having to be aware of the mechanics employed.

However, the nature of the different ways that master data are managed raises the question of consistency above the system level into the application service layer. This must be evaluated in the context of the degree of synchronization needed by the set of client applications using the master data asset. Therefore, the decisions regarding the selection of an underlying MDM architecture, the types of services provided, the application migration strategy, and the tools needed for integration and consolidation all depend on assessing the business process requirements (independently of the way the technology has been deployed!) for data consistency.

Each of the key characteristics impacts the maintenance of a consistent view of master data across the client application infrastructure, and each of the variant MDM architecture styles will satisfy demands across these different dimensions in different ways. Our goal is to first understand the measures of the dimensions of synchronization and how they address the consistency of master data. Given that understanding, the MDM architects would assess the applications that rely on the master data asset as well as the future requirements for applications that are planned to be developed and determine what their requirements are for master data synchronization. At this point, each of the architectural styles can be evaluated to determine how well each scores with respect to these dimensions. The result of these assessments provides input to inform the architect as to the best architecture to meet the organizational needs.

Timeliness

In its most general definition, timeliness refers to the appropriateness of when an event happens. In relation to data quality content, timeliness has been defined as the degree to which data represent reality from the required point in time (English, 1999). With regard to processing, timeliness (also referred to as latency) is associated with data availability, the degree to which customers have the data they need at the right time. Measures of timeliness can be made in relation to a set schedule or to the occurrence of an event. For example, timeliness is the degree to which data delivery from a source system conforms to a schedule for delivery. In large data assets, data is made available once processing is complete. Unsuccessful processing can result in delays in data availability. The timeliness of events within data processing itself can be a measure of the health of the data delivery mechanisms.

Data lag (the time between when data is updated in its source and when it is made available to data consumers), information float (the time between when a fact becomes known and when it is available for use) and volatility (the degree to which data is likely to change over time) all influence the perception of data currency and timeliness. Given these factors, for most people it’s clear that data can be “timely” according to a schedule and still not meet the needs of data consumers. In that case, the thing that needs to be changed is the schedule itself (for example, to reduce data lag), not how we objectively measure timeliness. As with other aspects of data, metadata about data timeliness—schedules and update processes—needs to be shared with data consumers so that they can make better decisions about whether and how to use data.

Timeliness

Almost all respondents noted that timeliness depended upon disciplinary context, especially upon the research question at hand. An older work, even in science, may still be timely if it is a work that has established a field or subfield. The senior scientist also noted that, if a set of research questions had fallen into neglect but was worth reviving, something only an expert in the field might know based on knowledge of content, the most timely source may be quite old. The senior social scientist was willing to look at some materials which were decades old. Both philosophers felt that one could easily reference material thirty to fifty years old.

What is ‘timeliness’, therefore, in practice? ‘Timely’ is what is taken to be relevant—it is not primarily defined by date of publication. It can only be determined by an understanding of the research questions existing in the discipline itself, or even in a subfield of a discipline. More often than not, a determination of what is ‘timely’ depends upon the specific research topic and can only be fully grasped within a disciplinary framework. This indicates that both the library faculty and students will be able to identify a timely work only if they know some of the content of a discipline.

12.2.5.1 Design responsiveness

Software design responsiveness involves the timeliness of the software product’s response to user inputs, external interface stimuli, or interactions with elements of the computing environment. The software structural design must be evaluated to determine if the design can be enhanced to improve the software product’s responsiveness to requested actions. The following guidelines are provided that address enhancing the software responsiveness to user-based requests5 :

Provide timely feedback concerning the requested action:

Promptly acknowledge a user input.

Provide data processing progress indicators for actions taking a significant amount of time.

Respond initially by providing the most important information then disclosing additional information when it becomes available.

Alert the user concerning the anticipated delay needed to respond to complicated requests.

Prioritize data processing actions:

Postpone low-priority data processing actions until computing resources are available.

Anticipate data processing needs and perform actions in advance, when possible.

Optimize task queue backlog:

Reorder the task queue based on priority.

Flush tasks that are overtaken by events or may no longer be needed.

Multitasking performance supervision:

Monitor multitasking progress and adjust resource allocations to optimize task execution and termination.

Balance task duration and resource commitments.

Predict task durations and determine task discreteness, concurrency, and synchronization tactics.

Establish resource monitoring and intercession supervision procedures by anticipating resource conflicts and deadlock situations.

Publisher Summary

Data quality ensures the accuracy and timeliness of data and it is much easier to ensure data quality before data get into a database than after they are stored. To be useful, the data in a database must be accurate, timely, and available when needed. Data quality problems arise from a wide range of sources and have many remedies. One source of data quality problems is missing data. There are two general sources—data that are never entered into the database and data that are entered but deleted when they shouldn't be. Another quality problem is that of incorrect data. Incorrect data is probably the worst type of problem to detect and prevent. Often the incorrect data aren't detected until someone external to an organization makes a complaint. Determining how the error occurred is equally difficult because sometimes the problems are one of a kind. Data can sometimes also be incomprehensible. Unlike incorrect data, it is relatively easy to spot incomprehensible data, although finding the source of the problem may be as difficult as it is with incorrect data. The last problem with data is inconsistency of data. If the data are to be consistent, then the name and address of a customer must be stored in exactly the same way in both databases.

Assessing the Validity of Data Content

While many measures of completeness and timeliness are interwoven with data processing, measures of validity are based on comparisons to a standard or rule that defines the domain of valid values. Most fields for which validity can be measured are populated with codes or other shorthand signifiers of meaning. Codes may be defined in a reference table or as part of a range of values or based on an algorithm. Conceptually, a validity measure is just a test of membership in a domain. From a technical point of view, ways of measuring validity depend on how the data that defines the domain is stored. Validity checks can be understood in terms of the way their domains are defined.

Basic validity check—Comparison between incoming values and valid values as defined in a code table or listing.

Basic range of values check—Comparison of incoming data values to values defined within a stated range (with potential for a dynamic range) including a date range.

Validity check based on a checksum or other algorithm—for example, to test the validity of Social Security numbers, National Provider Identifiers, or other numbers generated through an algorithm.

The question most people want to answer in relation to validity is: How much of this data is invalid? In some cases, domains contain a small set of values, and validity measures at the level of the individual value are comprehensible. For example, in most systems, the gender code consists of representations of male, female, and unknown. While the concept of gender in our culture is changing (so that some systems will have values to signify transgender or changes in gender), the set of codes values used to represent gender is still relatively small. On the other hand, codes representing medical procedures (CPT codes, HCPCS codes, ICD procedure codes) number in the tens of thousands. To make measurement of validity comprehensible (especially for high cardinality code sets), it is helpful to have both detail of the individual codes and a measure of the overall level of valid and invalid codes.

When we first defined the DQAF, we included pairs of validity measures: a detail measure at the level of the value and a summary accounting for the roll-up of counts and percentages of valid and invalid codes. However, it quickly became clear that the validity summary measure looked pretty much the same (and thus could be represented the same way) regardless of the basis of the detailed validity measure. So, we created one type to describe all validity roll-ups.

Roll-up counts and percentage of valid/invalid from a detailed validity measurement.

Validity checks against code tables or ranges of values focus on one field at a time. More complex forms of validity involve the relationship between fields or across tables. These can be more challenging to measure because they often require documentation of complex rules in a form that can be used to make the comparisons.

Validity of values in related columns on the same table; for example ZIP codes to State Abbreviation Codes.

Validity of values based on a business rule; for example, Death Date should be populated only when Patient Status equals “deceased”.

Validity of values across tables; for example, ZIP Code to State Code on two different tables.

As with the single-column measures of validity, each of these validity measurement types should include a roll-up of valid-to-invalid results (see Figure 6.2). In many cases, because of differences in the timing of data delivery and the nature of the data content, cross-table relationships cannot be measured effectively through in-line measures. These are better thought of as measures of integrity that can be taken as part of periodic measurement of data.

2.1.3 Analytical Complexity

Due to the availability of large volumes of data with high velocity and variety, businesses demand different and more complex analytical tasks to produce the insight required to solve their business problems. Some of these analyses require that the data be prepared in a fashion not foreseen by the original data warehouse developers. For example, the data that should be fed into a data mining algorithm should have different characteristics regarding the variety, volume and velocity of data.

Consider the example of retail marketing: the campaign accuracy and timeliness need to be improved when moving from retail stores to online channels where more detailed customer insights are required [11]:

In order to determine customer segmentation and purchase behavior, the business might need historical analysis and reporting of customer demographics and purchase transactions

Cross-sell opportunities can be identified by analyzing market baskets that show products that can be sold together

To understand the online behavior of their customers, click-stream analysis is required. This can help to present up-sell offers to the visitors of a Web site

Given the high amount of social network data and user-generated content, businesses tap into the data by analyzing product reviews, ratings, likes and dislikes, comments, customer service interactions, and so on.

These examples should make it clear that, in order to solve such new and complex analytical tasks, data sources of varying complexity are required. Also, mixing structured and unstructured data becomes more and more common [11].