Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Article . 2023
License: CC BY
Data sources: Datacite
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Article . 2023
License: CC BY
Data sources: ZENODO
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Article . 2023
License: CC BY
Data sources: Datacite
versions View all 2 versions
addClaim

BIG DATA PREPROCESSING USING ENHANCED DATA QUALITY RULES DISCOVERY MODEL (EDQRM)

Authors: K. Dharani* & Dr. G. Abel Thangaraja**;

BIG DATA PREPROCESSING USING ENHANCED DATA QUALITY RULES DISCOVERY MODEL (EDQRM)

Abstract

{"references": ["1.\tDai, H., Zhang, S., Wang, L., & Ding, Y. (2016). Research and implementation of big data preprocessing system based on Hadoop. 2016 IEEE International Conference on Big Data Analysis (ICBDA). 2.\tWang, Z., Huang, X., Song, Y., & Xiao, J. (2017). An outlier detection algorithm based on the degree of sharpness and its applications on traffic big data preprocessing. 2017 IEEE 2nd International Conference on Big Data Analysis (ICBDA) 3.\tCelik, O., Hasanbasoglu, M., Aktas, M. S., Kalipsiz, O., & Kanli, A. N. (2019). Implementation of Data Preprocessing Techniques on Distributed Big Data Platforms. 2019 4th International Conference on Computer Science and Engineering (UBMK). 4.\tLincy, S. S. B. T., & Kumar, N. S. (2017). An enhanced pre-processing model for big data processing: A quality framework. 2017 International Conference on Innovations in Green Energy and Healthcare Technologies (IGEHT). 5.\tP. Glowalla, P. Balazy, D. Basten, and A. Sunyaev, \"Process-Driven Data Quality Management - An Application of the Combined Conceptual Life Cycle Model,\" in 2014 47th Hawaii International Conference on System Sciences (HICSS), 2014, pp. 4700-4709. 6.\tF. Sidi, P. H. Shariat Panahy, L. S. Affendey, M. A. Jabar, H. Ibrahim, and A. Mustapha, \"Data quality: A survey of data quality dimensions,\" in 2012 International Conference on Information Retrieval Knowledge Management (CAMP), 2012, pp. 300-304. 7.\tY. W. Lee, \"Crafting rules: context-reflective data quality problem solving,\" J. Manag. Inf. Syst., vol. 20, no. 3, pp. 93-119, 2003. 8.\tP. Z. Yeh and C. A. Puri, \"An Efficient and Robust Approach for Discovering Data Quality Rules,\" 22nd IEEE International Conference on Tools with Artificial Intelligence (ICTAI), 2010, vol. 1, pp. 248-255. 9.\tF. Chiang and R. J. Miller, \"Discovering data quality rules,\" Proc. VLDB Endow., vol. 1, no. 1, pp. 1166- 1177, 2008. 10.\tW. Fan, \"Dependencies revisited for improving data quality,\" in Proceedings of the twenty-seventh ACM SIGMOD-SIGACT-SIGART, 2008, pp. 159-170."]}

In the Big Data Era, data is the center for any governmental, institutional, and private organization. Endeavors were equipped towards extricating profoundly important bits of knowledge that can't occur assuming data is of low quality. Hence, data quality (DQ) is considered as a vital component in big data processing. In this stage, bad quality data isn't entered to the Big Data value chain. This paper, proposed the Enhanced data quality Rules discovery model (EDQRM) for assessment of quality and Big Data pre-processing. EDQRM discovery model to improve and precisely focus on the pre-processing exercises in view of quality requirements. Characterized, a bunch of pre-processing exercises related with data quality dimensions (DQD's) to automatize the EDQRM process. Rules improvement is applied on approved rules to stay away from multi-passes pre-processing exercises and disposes of copy rules. Directed tests showed an expanded quality scores in the wake of applying the found and optimized EDQRM's on data.

Keywords

Big Data; Data Quality Evaluation; Data Quality Rules Discovery; Big Data Pre-Processing

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
    OpenAIRE UsageCounts
    Usage byUsageCounts
    visibility views 3
    download downloads 3
  • 3
    views
    3
    downloads
    Powered byOpenAIRE UsageCounts
Powered by OpenAIRE graph
Found an issue? Give us feedback
visibility
download
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
views
OpenAIRE UsageCountsViews provided by UsageCounts
downloads
OpenAIRE UsageCountsDownloads provided by UsageCounts
0
Average
Average
Average
3
3
Green