Abstract: This

article aimed to offer students and

researchers how to analyze

validity and reliability of their questionnaire as the instrument of research

or any other evaluation purposes. Analyzing

validity and reliability of questionnaire is really important to make sure that

the questionnaire is compatible and credible to be the instrument. The basis of

this writing was based on a mini-research with qualitative method. The subject

of research was the students of English Department who have written their

proposal and have conducted examination of their thesis. The sources of the

data were from observation and interview. The result was found

that many students did not know well about how to analyze the validity and

reliability of the questionnaire; they also did not know the significance of the

analysis; as well as they did

not know how to determine the appropriate technique and formula of the

analysis. Therefore, in this paper, it is offered how to analyze the validity

and reliability of Likert scale questionnaire. The popular technique of validity analysis is using

item analysis with correlation formula, and the reliability is using Cronbach Alpha

formula. The formulas are considered based on the types of data collection.

Keywords:

validity, reliability, instrument, questionnaire, research, evaluation

1.

Introduction

Evaluation and research are the two important issues

in education. In evaluation and research, there are various instruments those

can be used to get data; one of them is questionnaire. Questionnaire is a

series of written questions that should be answered by respondents in order to

get necessary information. Nigel, Fox, & Hunn (2009)

explained that questionnaire is as a tool of data collection on a research that

is often used in survey. Many types of questionnaires, but in this discussion

the writer only focuses on the popular and easy one; it is Likert scale

questionnaire. Likert scale questionnaire is a questionnaire where the form of

the data is ordinal (Norman, 2010).

Although the questionnaire is possible to be used, the

questionnaire is not really popular and familiar to be the instrument of

classroom assessment/ evaluation; test is the popular one; especially written

test. The questionnaire is very popular in a research. It is often used for

survey both qualitative and quantitative research. Such as Bird (2009)

mentioned that questionnaires are popular and fundamental tools for

acquiring information on public knowledge and perception.

As usual, instrument of a research (both

test and questionnaire) should be valid and reliable. So, in this case valid

and reliable questionnaire is a must. Valid questionnaire means that the

questionnaire can function as an instrument to measure what it is intended to

measure. This refers to some researchers’ statements talking about the term

validity such as Hughes (2003, p. 26) stated if a

test is said valid if it measures accurately what it is intended to measure. Also Rajhy (2014) stated that validity refers to the extent to

which the test measures what it says it measures. In line with those writers, Sugianto (2017) wrote validity is the extent to which

the test measures what it wants

to measure. Hamzah & Abdullah (2011)

wrote that the validity of test in the context of

testing the performance of students refers to the compatibility of the

aim of test with the contents of test. In

other words, a valid test can really measure what it is supposed to measure. So it can be concluded that validity of

questionnaire refers

to the suitability between a questionnaire as

an instrument of research and

the domain of what it is supposed to investigate

or measure.

The

extent of validity can be analyzed logically, empirically,

and item-based validity (Sugianto, 2017). For a research, it is popular to analyze the validity

of questionnaire based on items of the questionnaire. (Sugianto, 2016b) in his

paper talking about item validity stated that item validity meant

the accuracy of each item to measure what it supposed to measure; and the analysis is done by correlating the scores of each item to the total score. So the

technique of item analysis between a test and questionnaire is the same; it is

by correlation.

Reliable

questionnaire means that the questionnaire shows its credibility to be the

instrument of data collection. In a research, reliability level checking is the

second consideration to prove that the instrument is proper as the instrument

of the research. In term of test and it can be applied for term of

questionnaire, Sugianto (2017) explained

that reliability

refers to the consistency of scores, so it means a test would be considered as a

good test and credible as a measurement of learning achievement if the

result/score of test is consistent in how many times the test conducted. Brown (2005, p. 175) also

showed that

reliability is defined as the extent to which the result can be considered

consistent or stable. Then

Rajhy (2014) stated

that the results of test should be consistent where they remain stable and the

test should not produce different results when it is used in different days. So,

a questionnaire is considered reliable if the scores are approximately the same repeatedly. For the Likert scale questionnaire, since

the score is gotten from scale (ordinal number) there is a popular technique

and formula that can be used; it is Cronbach alpha. Alpha was developed

by Lee Cronbach in 1951 to provide a measure of the internal consistency of a test or scale (Tavakol & Dennick, 2011). Now it is popular to be called Cronbach alpha. Taber (2017) said that Cronbach’s alpha is a statistic commonly quoted by authors to

demonstrate that tests and scales that have been constructed or adopted for research

projects are fit for purpose. Cronbach alpha is popular in a research

to be used to see the reliability of questionnaire and essay test.

This paper was inspired based on the writer’s

experience become advisor of thesis writing where many students did not know

well how to determine the instrument of their research; they did not know the

function of try out; they did not know well how to see the validity and reliability

of their instrument either for test or questionnaire; they did not know how to

determine the appropriate and correct technique and formula of the validity and

reliability analysis of their instrument; and so on. So, in this paper the

writer needs to answer some questions; they are: 1) How are the knowledge of

students related to validity, reliability, and questionnaire? 2) How to analyze

the validity and reliability of questionnaire in appropriate way?

The objectives of this writing are to offer students

and researchers how to analyze validity and reliability of their questionnaire

as the instrument of research. The significance is that as reference for the

students and researchers so they are able to determine the appropriate and easy

technique and formula to analyze the validity and reliability of the

questionnaire.

Drost (2011) actually had offered generally about

technique and formula those could be used to see the extent of validity and

reliability, but there was no discussion about types of data as the basis of

technique and formula used. His discussion was in general and not specifically

mentioned about questionnaire matter.

In previous research and paper, actually the writer

had a research related to validity and reliability of instrument such as Sugianto (2011) that discussed about validity and reliability

of formative test, Sugianto (2016a) that discussed about validity and reliability

of English National Final Examination test, and also Sugianto (2017) that discussed about validity and reliability

of summative test. All the researches above are only all about instruments in

form of test for classroom matter. So there are no discussions related to

analysis for instrument of research matter. However, actually the procedures

are applicable for research matter as long as the form of the instrument is

also test like the test analyzed. Another previous writing is Sugianto (2016b) which discussed about the technique and

procedure of item analysis of objective test. This paper is type of item

analysis but special about objective test. The technique and procedure are

applicable for classroom matter test and research matter test/ instrument.

2.

Methodology

As written in the introduction

that the students still have problem in their research related to their

instrument, to help the students’ problem the writer conducted this

mini-research. The methodology in this mini research was conducted in order to

know the fact exactly their condition or knowledge especially about their

ability in analyzing the questionnaire in terms of validity and reliability. Therefore,

in collecting the data the writer used observation and interview as the

instrument. The subject of study is some students of English Department of

State Islamic Institute of Palangka Raya Indonesia who have written their proposal of thesis writing and have conducted

examination of their thesis. The data were

analyzed qualitatively.

After knowing the condition or knowledge of students

related to their ability and knowledge about instrumentation of research, the

writer would elaborate how to analyze the validity and reliability of the

questionnaire especially Likert scale questionnaire.

3.

Finding and Discussion

As stated in

introduction, actually the writer got the overview of students from the

writer’s experience became advisor of thesis writing. Then it was investigated

more by using procedure as stated in the methodology to find the real facts. From

the observation when the thesis examination and interview to some students, it

was found that although almost all of

them know what questionnaire is, but they did not know well how to see or

analyze the validity and reliability of the questionnaire; they did not know

the significance of analyzing validity and reliability of the questionnaire as

an instrument; as well as they did not know how to determine the appropriate

technique and formula of the validity and reliability analysis of the

questionnaire.

Based on the finding above it was needed to elaborate

the procedure, technique and formula to analyze the validity and reliability of

the questionnaire. The elaborations are in the following discussion.

3.1. Validity of

Questionnaire

The easiest procedure to analyze the validity of

questionnaire is based on items. Every item of questionnaire is analyzed to see

the validity of each item. The technique is by correlating the score of item to

the total score. So, it would use correlation formula. The popular one is

Product Moment correlation or also mentioned as Pearson Product Moment.

Determining Product Moment correlation is

actually based on the types of data. Actually there are some types of

correlation formula such as Pearson

correlation (Product Moment, Point-Biserial, etc.), Kendall rank correlation, and

Spearman rank

correlation. Types of data influence the types of

correlation formula that is appropriate to be used. Sugianto (2016b) wrote that the appropriate formula in analyzing the

data related closely to the types of data. For example if the data those would be correlated are continuous data/

variables (ordinal, interval, and ratio), the appropriate formula is Product

Moment correlation formula. But if one of the data is nominal (dichotomous),

Point-Biserial correlation formula is the best one (Brown, 2005). Hamzah & Abdullah (2011) also discussed about the concept of the relation about types of data to

the statistical measurement.

Especially for questionnaire, based on the types of data those would be

correlated which are continuous data, so it would be Product Moment correlation

formula. Product Moment formula can be calculate manually by following the formula and can also be

processed by computer software such as Microsoft Excel and SPPS. The formula is actually as follow:

Where:

N : number of pairs of

scores

:

sum of the products of paired scores

:

sum of X scores

:

sum of Y scores

:

sum of squared X scores

:

sum of squared Y scores

Interpretation of coefficient could consult to r-table. If the

coefficient r of the item is equal or more than the value of r-table (r ?

r-table), it means the item is valid. But if the coefficient r of

the item is less than the value of r-table (r