Environment and Social Psychology

The development of an analysis of quantitative measurements competence disparity scale for graduate students

Jian-Hong Ye

Faculty of Education, Beijing Normal University

Yuting Cui

College of Education Cadre, Beijing Institute of Education

Weiguaju Nong

School of Education, Guangxi University of Foreign Languages

Yi-Sang Lee

Department of Industrial Education, National Taiwan Normal University

Xiantong Yang

Faculty of Psychology, Beijing Normal University

Li Wang

College of Education and Music, Hainan Vocational University of Science and Technology

Jhen-Ni Ye

Graduate Institute of Technological & Vocational Education, National Taipei University of Technology

DOI: https://doi.org/10.54517/esp.v9i6.2576

---

Abstract

Academic research competencies play the most essential role in graduation for graduate students, but there is still a lack of in-depth research on this matter. Since talent development suggestions are provided based on the survey results of capability gaps, this study aimed to develop a scale to examine the research competency disparity between graduation requirements and graduate students’ current levels of competency in the context of quantitative research in the education, arts and humanities, and business domains. It is named the “Quantitative Academic Competence Scale”. To explore the needs of graduate students regarding their academic research competence, we constructed a draft questionnaire with nine categories of competence; academic experts validated the scale. It was then administered to graduate students to explore their perceived disparities. A total of 457 participants assisted in filling out the questionnaire, from which 86 invalid samples were deleted, leaving 371 valid participants, giving an effective recovery rate of 81.2%. After collection and analysis, the analysis results found that the three graduate students’ academic research competences with the greatest disparity were English writing competence, research design and implementation competence, and data processing and analyzing competence. According to the results, graduate students should take more courses on English writing competence, research design and implementation competence, and data processing and analyzing competence to enhance their competencies. generative artificial intelligence (GenAI) suggests that research ability factors can be comprehensively considered. Based on the graduate students’ ability gaps, curriculum settings and training plans can be adjusted and more targeted (personalized) at teaching and guidance to help graduate students improve their academic performance and reduce weaknesses in their research capabilities.

---

References

[1]. Xu, P. H., & Zhong, Z. C. (2013). Multiple mentors’ program of graduate education: Enlightenment from the graduate education mode of University of Mississippi. Journal of Architectural Education in Institutiong of Higher Learning, 22(4), 34-36.

[2]. Zhou, Y. W. (2006). The analysis concept of data analysis in the era of big data. Forestry Education in China, 2006(6), 43-45.

[3]. Zhu, Y. (2003). Academic writing norms and graduate students' academic research competency. Foreign Languages and Their Teaching, 172, 25-27.

[4]. Hao, Y. W. (2013). Common mistakes made in graduate theses. Taiwan Educational Review Monthly, 2(3), 24.

[5]. Aitchison, C., & Lee, A. (2006). Research writing: Problems and pedagogies. Teaching in Higher Education, 11(3), 265-278. https://doi.org/10.5430/ijhe.v4n1p61

[6]. Sun, G. (2007). Thesis topic resource, method and technique of postgraduates' dissertations. Journal of Shanxi University of Finance and Economics (Higher educatopn), 10(3), 21-27.

[7]. Hong, J. C., Horng, J. S., ChanLin, L. J., & Lin, C. L. (2008). Competency disparity between pre-service teacher education and in-service teaching requirements in Taiwan. International Journal of Educational Development, 28(1), 4-20.

[8]. Tsai, Y. H., Hong, J. C., & Yu, X. X. (2019). Competence disparity of international Chinese teachers and PCK questionnaire: From the international baccalaureate perspective. Psychological Testing, 66(4), 403-428.

[9]. Demetriou, A., Kazi, S., Makris, N., & Spanoudis, G. (2020). Cognitive competency, cognitive self-awareness, and school performance: From childhood to adolescence. Intelligence, 79, 101432. https://doi.org/10.1016/j.intell.2020.101432

[10]. Huang, C. H. (2016). Reflection of quantitative and qualitative research. Taiwan Educational Review Monthly, 5(9), 149-154.

[11]. Hsu, C. L. (2018). Learning dilemmas and coping strategies of thesis writing for graduate students (Unpublished doctoral dissertation). National Pingtung University, Pingtung County.

[12]. Hong, J. C., Hwang, M. Y., Hsu, L. C., & Lee, C. K. (2009). Perceived competency disparities between pre-service training and job demands of primary school administrators. Journal of Research in Education Sciences, 54(3), 29-51.

[13]. Hicks, P. J., Schumacher, D., Guralnick, S., Carraccio, C., & Burke, A.E. (2014). Domain of competence: Personal and professional development. Academic Pediatrics, 14, 80-97.

[14]. Campion, M. A., Fink, A. A., Ruggeberg, B. J., Carr, L., Phillips, G. M., & Odman, R. B. (2011). Doing competencies well: Best practices in competency modeling. Personnel Psychology, 64, 225-262. https://doi.org/10.1111/j.1744-6570.2010.01207.x

[15]. Catano, V. M., Wiesner, W. H., & Hackett, R. D. (2012). Recruitment and selection in Canada (5th ed.). Toronto: Nelson Canada.

[16]. Vidal-Gomel, C., & Samurçay, R. (2002). Qualitative analyses of accidents and incidents to identify competencies: The electrical systems maintenance case. Safety Science, 40(6), 479-500. https://doi.org/10.1016/S0925-7535(01)00016-9

[17]. Stagg, A., & Kimmins, L. (2014). First year in higher education (FYHE) and the coursework post-graduate student. The Journal of Academic Librarianship, 40(2), 142-151.

[18]. Wu, Z. Y. (2007). Reflections on cultivating graduate students' academic innovation capacity. Academic Degrees & Graduate Education, 11, 19-23.

[19]. OuYang, X. L., & Guo, X. Q. (2009). Postgraduates’ writing of thesis and academic papers. China Electric Power Education, 143, 40-41.

[20]. Pan, H. L. (2003). Changes in the paradigm of social science research. Educational Research & Information, 11(1), 111-143.

[21]. Bi, H. D. (2012). Why didn’t professor tell me that (full edition 2010). New Taipei: Bihspace.

[22]. Guo, S. Y. (2012). Psychological and Educational Research Method. Taichung: JingHua.

[23]. Grimm, L. J., & Harvey, J. A. (2022). Practical steps to writing a scientific manuscript. Journal of Breast Imaging, 4(6), 640-648. https://doi.org/10.1093/jbi/wbac059

[24]. Holliday, A. (2007). Doing & writing qualitative research (2nd ed.). London: Sage.

[25]. Redman, P., & Maples, W. (2017). Good essay writing: a social sciences guide. New York: Sage.

[26]. Nayak, U. Y., Hoogar, P., Mutalik, S., Udupa, N. (2023). Writing a postgraduate or Foctoral Thesis: A Step-by-Step Approach. In G. Jagadeesh, P. Balakumar, & F. Senatore (Eds.), The quintessence of basic and clinical research and scientific publishing (pp. 769-781). Springer, Singapore. https://doi.org/10.1007/978-981-99-1284-1_48

[27]. Wu, P. Y. (2016). Are you listening when teacher is talking? Standard and format of academic writing (The 3rd print). Taipei: Hanlu.

[28]. Yan, Z. L. (2018). Even a fool can write a thesis (Quantitative + qualitative revised edition): A guide to writing social science theses and dissertations. Taipei: Wunan.

[29]. Davis III, T. H., Wagner, G. S., Gleim, G., Andolsek, K. M., Arheden, H., Austin, R., ... & Noga Jr, E. M. (2006). Problem-based learning of research skills. Journal of Electrocardiology, 39(1), 120-128. https://doi.org/10.1016/j.jelectrocard.2005.06.107

[30]. Hung, R. Y. Y., Lien, B. Y. H., Fang, S. C., & McLean, G. N. (2010). Knowledge as a facilitator for enhancing innovation performance through total quality management. Total Quality Management, 21(4), 425-438. https://doi.org/10.1080/14783361003606795

[31]. Zhu, H., Li, W. L., & Zuo, Z. J. (2011). Graduate students’ creativity in China colleges and universities: Analysis on factors and mechanisms. Journal of Higher Education, 9(1), 74-82.

[32]. Asiyai, R. I. (2022). Best practices for quality assurance in higher education: implications for educational administration. International Journal of Leadership in Education, 25(5), 843-853.. https://doi.org/10.1080/13603124.2019.1710569

[33]. Hong, J. C. (2003). Knowledge innovation and learning organization. Taipei: Wunan.

[34]. Chen, X., Yang, C. Q., & Huang, S. M. (2013). An exploration of improper citations and plagiarism patterns of theses and dissertations in Taiwan: Taking the “Review of literatures” as examples. The Journal of Information Societyccis, 24, 74-119. https://doi.org/10.29843/JCCIS.201301_(24).0004

[35]. Schneider, K.P. (2018). Methods and ethics of data collection. In: A.H. Jucker, K.P. Schneider, & W. Bublitz, (Eds.), Methods in pragmatics (pp. 37-93). Berlin: De Gruyter Mouton.

[36].

[37]. Locher, M. A., & Bolander, B. (2019). Ethics in pragmatics. Journal of Pragmatics, 145, 83-90. https://doi.org/10.1016/j.pragma.2019.01.011

[38]. Cargill, M., O’Connor, P., & Li, Y. (2012). Educating Chinese scientists to write for international journals: Addressing the divide between science and technology education and English language teaching. English for Specific Purposes, 31(1), 60-69. https://doi.org/10.1016/j.esp.2011.05.003

[39]. Belcher, D. D. (2007). Seeking acceptance in an English-only research world. Journal of Second Language Writing, 16(1), 1-22. https://doi.org/10.1016/j.jslw.2006.12.001

[40]. Ceng, J. B., Liao, W. W., Xian, M. H., & Lu, W. L. (2013). An analysis on postgraduates’ needs of academic English. China University Teaching, 2013(10), 79-83.

[41]. Ebrahimi, S. F., & Khedri, M. (2015). Thematicity in research article abstracts: A cross-disciplinary study. Educational Quest: An International Journal of Education and Applied Social Sciences, 2(3), 287-292.

[42]. Chen, Q. (2019). Theme-Rheme structure in Chinese doctoral students' research writing: From the first draft to the published paper. Journal of English for Academic Purposes, 37, 154-167. https://doi.org/10.1016/j.jeap.2018.12.004

[43]. Nesi, H., & Gardner, S. (2012). Genres across the disciplines: Student writing in higher education. Cambridge: Cambridge University Press.

[44]. Cumming, A., Lai, C., & Cho, H. (2016). Students' writing from sources for academic purposes: A synthesis of recent research. Journal of English for Academic Purposes, 23, 47-58. https://doi.org/10.1016/j.jeap.2016.06.002

[45]. Aitchison, C. (2009). Writing groups for doctoral education. Studies in Higher Education, 34(8), 905-916. https://doi.org/10.1080/03075070902785580

[46]. Bolderston, A. (2008). Writing an effective literature review. Journal of Medical Imaging and Radiation Science, 39(2), 85-92. https://doi.org/10.1016/j.jmir.2008.04.009

[47]. Singh, R., & Kumar, S. (2019). Information literacy competency level of social science researchers with respect to information use ethics: A study. DESIDOC Journal of Library & Information Technology, 39(2), 101-108.

[48]. Lovitts, B. E. (2007). Making the implicit explicit: Creating performance expectations for the dissertation. Sterling: Stylus.

[49]. Cooper, D. R., & Schindler, P. S. (2006). Business research methods (9 th edition). New York: McGraw-Hill.

[50]. Sekaran, U., & Bougie, R. (2010). Research methods for business: A skill building approach (5th ed.). Hoboken: John Wiley and Sons.

[51]. Bao, S. M. (2012). A comparative study of Chinese and American systems of producing doctors in higher pedagogy: A case study of Xiamen university and Stanford university. Journal of Higher Education Management, 6(4), 59-66.

[52]. Zhu, J., Zhang, G., & Liu, X. (2014). Clarity of a philosophy of data analysis during the age of big data. Statistical Research, 31(2), 10-17.

[53]. Zhong, H., Li, J., & Cao, H. (2012). The exploration of constructing the whole progressive cultivation mode for the innovative practical competency of the graduate students of science and engineering. Journal of Higher Education Research, 35(1), 54-62.

[54]. Jung, Y. M. (2019). Data analysis in qualitative research. In P. Liamputtong (Ed.), Handbook of research methods in health social sciences (pp.955-969). Singapore: Springer. https://doi.org/10.1007/978-981-10-5251-4_109

[55]. Saunders, M., Lewis P., & Thornhill, A. (2009). Research Methods for Business Students (5thedition). Upper Saddle River: Prentice Hall.

[56]. Zyphur, M. J., & Pierides, D. C. (2017). Is quantitative research ethical? Tools for ethically practicing, evaluating, and using quantitative research. Journal of Business Ethics, 143(1), 1-16. https://doi.org/10.1007/s10551-017-3549-8

[57]. Ben-Zvi, D., & Garfield, J. B. (2004). Statistical literacy, reasoning, and thinking: Goals, definitions, and challenges. In D. Ben-Zvi, & J. Garfield (Eds.), The challenge of developing statistical literacy, reasoning and thinking (pp. 3-15). Dordrecht: Springer. https://doi.org/10.1007/1-4020-2278-6_1

[58]. Zhou, X. N., Zhang, Z. X., & Qiu, R. Z. (2008). Discussion on approaches of cultivating postgraduates’ research capacity. Forestry Education in China, 2008(3), 44-46.

[59]. Xu, L. R. (2003). Questions and suggestions on writing science and education papers. Guidance of Elementary Education, 43(2), 24-28. https://doi.org/10.6772/GEE.200312.0024

[60]. Huang, L., & Pashler, H. (2005). Attention capacity and task difficulty in visual search. Cognition, 94(3), 101-111. https://doi.org/10.1016/j.cognition.2004.06.006

[61]. Bodie, G. D. (2010). A racing heart, rattling knees, and ruminative thoughts: Defining, explaining, and treating public speaking anxiety. Communication Education, 59(1), 70-105. https://doi.org/10.1080/03634520903443849.

[62]. van Ginkel, S., Gulikers, J., Biemans, H., & Mulder, M. (2015). Towards a set of design principles for developing oral presentation competence: A synthesis of research in higher education. Educational Research Review, 14, 62-80. https://doi.org/10.1016/j.edurev.2015.02.002

[63]. Liu, H. K., & Huang, L. (2011). Innovating cultivation of contemporary graduate students’ comprehensive competency. China Electric Power Education, 194, 45-46.

[64]. Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2019). Multivariate data analysis (8th ed.). Cengage.

[65]. Lei, M., & Lomax, R. G. (2005). The effect of varying degrees of non-normality in structural equation modeling. Structural Equation Modeling: A multidisciplinary Journal, 12(1), 1-27. https://doi.org/10.1207/s15328007sem1201_1

[66]. Carter, S. R. (2016). Using confirmatory factor analysis to manage discriminant validity issues in social pharmacy research. International Journal of Clinical Pharmacy, 38(3), 731-737. https://doi.org/10.1007/s11096-016-0302-9

[67]. Kenny, D. A., Kaniskan, B., & McCoach, D. B. (2015). The performance of RMSEA in models with small degrees of freedom. Sociological Methods & Research, 44(3), 486-507. https://doi.org/10.1177/0049124114543236

[68]. Cor, M. K. (2016). Trust me, it is valid: Research validity in pharmacy education research. Currents in Pharmacy Teaching and Learning, 8(3), 391-400. https://doi.org/10.1016/j.cptl.2016.02.014

[69]. Green, S. B., & Salkind, N. (2004). Using SPSS for Windows and Macintosh: Analyzing and understanding data (4th ed.). Prentice-Hall.

[70]. Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273-1296.

[71]. Duxbury, J. (2003). Testing a new tool: The management of aggression and violence attitude scale (MAVAS). Nurse Researcher, 10(4), 113-127. https://doi.org/10.7748/nr2003.07.10.4.39.c5906

[72]. Farrell, A. M. (2010). Insufficient discriminant validity: A comment on Bove, Pervan, Beatty, and Shiu (2009). Journal of Business Research, 63(3), 324-327. https://doi.org/10.1016/j.jbusres.2009.05.003

[73]. Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115-135. https://doi.org/10.1007/s11747-014-0403-8

[74]. Awang, Z. (2015). SEM made simple, a gentle approach to learning structural equation modeling. Selangor, MY: MPWS Rich Publication Sdn. Bhd.

[75]. Ahmad, S., Zulkurnain, N. N. A., & Khairushalimi, F. I. (2016). Assessing the validity and relicompetency of a measurement model in structural equation modeling (SEM). Journal of Advances in Mathematics and Computer Science, 15(3), 1-8. https://doi.org/10.9734/BJMCS/2016/25183

[76]. Rönkkö, M., & Cho, E. (2022). An updated guideline for assessing discriminant validity. Organizational Research Methods, 25(1), 6-14. https://doi.org/10.1177/1094428120968614

[77]. Morin, A. (2006). Levels of consciousness and self-awareness: A comparison and integration of various neurocognitive views. Consciousness and Cognition, 15 (2), 358-371. https://doi.org/10.1016/j.concog.2005.09.006

[78]. Lage, C. A., Wolmarans, D. W., & Mograbi, D. C. (2022). An evolutionary view of self-awareness. Behavioural Processes, 194, 104543. https://doi.org/10.1016/j.beproc.2021.104543.

[79]. Han, S. (2003). An analysis and study of the graduate students' English writing competency. Journal of Shanxi Teachers University (Social Science Edition), 30(2), 141-144.

[80]. Downs, D., & Wardle, E. (2007). Teaching about writing, righting misconceptions:(Re) envisioning" first-year composition" as" introduction to writing studies". College Composition and Communication, 58(4), 552-584. https://doi.org/10.2307/355380