Abstract: This

article aimed to offer students and

researchers how to analyze

validity and reliability of their questionnaire as the instrument of research

or any other evaluation purposes. Analyzing

validity and reliability of questionnaire is really important to make sure that

the questionnaire is compatible and credible to be the instrument. The basis of

this writing was based on a mini-research with qualitative method. The subject

of research was the students of English Department who have written their

proposal and have conducted examination of their thesis. The sources of the

data were from observation and interview. The result was found

that many students did not know well about how to analyze the validity and

reliability of the questionnaire; they also did not know the significance of the

analysis; as well as they did

not know how to determine the appropriate technique and formula of the

analysis. Therefore, in this paper, it is offered how to analyze the validity

and reliability of Likert scale questionnaire. The popular technique of validity analysis is using

item analysis with correlation formula, and the reliability is using Cronbach Alpha

formula. The formulas are considered based on the types of data collection.

Keywords:

validity, reliability, instrument, questionnaire, research, evaluation

1.

Introduction

Evaluation and research are the two important issues

in education. In evaluation and research, there are various instruments those

can be used to get data; one of them is questionnaire. Questionnaire is a

series of written questions that should be answered by respondents in order to

get necessary information. Nigel, Fox, & Hunn (2009)

explained that questionnaire is as a tool of data collection on a research that

is often used in survey. Many types of questionnaires, but in this discussion

the writer only focuses on the popular and easy one; it is Likert scale

questionnaire. Likert scale questionnaire is a questionnaire where the form of

the data is ordinal (Norman, 2010).

Although the questionnaire is possible to be used, the

questionnaire is not really popular and familiar to be the instrument of

classroom assessment/ evaluation; test is the popular one; especially written

test. The questionnaire is very popular in a research. It is often used for

survey both qualitative and quantitative research. Such as Bird (2009)

mentioned that questionnaires are popular and fundamental tools for

acquiring information on public knowledge and perception.

As usual, instrument of a research (both

test and questionnaire) should be valid and reliable. So, in this case valid

and reliable questionnaire is a must. Valid questionnaire means that the

questionnaire can function as an instrument to measure what it is intended to

measure. This refers to some researchers’ statements talking about the term

validity such as Hughes (2003, p. 26) stated if a

test is said valid if it measures accurately what it is intended to measure. Also Rajhy (2014) stated that validity refers to the extent to

which the test measures what it says it measures. In line with those writers, Sugianto (2017) wrote validity is the extent to which

the test measures what it wants

to measure. Hamzah & Abdullah (2011)

wrote that the validity of test in the context of

testing the performance of students refers to the compatibility of the

aim of test with the contents of test. In

other words, a valid test can really measure what it is supposed to measure. So it can be concluded that validity of

questionnaire refers

to the suitability between a questionnaire as

an instrument of research and

the domain of what it is supposed to investigate

or measure.

The

extent of validity can be analyzed logically, empirically,

and item-based validity (Sugianto, 2017). For a research, it is popular to analyze the validity

of questionnaire based on items of the questionnaire. (Sugianto, 2016b) in his

paper talking about item validity stated that item validity meant

the accuracy of each item to measure what it supposed to measure; and the analysis is done by correlating the scores of each item to the total score. So the

technique of item analysis between a test and questionnaire is the same; it is

by correlation.

Reliable

questionnaire means that the questionnaire shows its credibility to be the

instrument of data collection. In a research, reliability level checking is the

second consideration to prove that the instrument is proper as the instrument

of the research. In term of test and it can be applied for term of

questionnaire, Sugianto (2017) explained

that reliability

refers to the consistency of scores, so it means a test would be considered as a

good test and credible as a measurement of learning achievement if the

result/score of test is consistent in how many times the test conducted. Brown (2005, p. 175) also

showed that

reliability is defined as the extent to which the result can be considered

consistent or stable. Then

Rajhy (2014) stated

that the results of test should be consistent where they remain stable and the

test should not produce different results when it is used in different days. So,

a questionnaire is considered reliable if the scores are approximately the same repeatedly. For the Likert scale questionnaire, since

the score is gotten from scale (ordinal number) there is a popular technique

and formula that can be used; it is Cronbach alpha. Alpha was developed

by Lee Cronbach in 1951 to provide a measure of the internal consistency of a test or scale (Tavakol & Dennick, 2011). Now it is popular to be called Cronbach alpha. Taber (2017) said that Cronbach’s alpha is a statistic commonly quoted by authors to

demonstrate that tests and scales that have been constructed or adopted for research

projects are fit for purpose. Cronbach alpha is popular in a research

to be used to see the reliability of questionnaire and essay test.

This paper was inspired based on the writer’s

experience become advisor of thesis writing where many students did not know

well how to determine the instrument of their research; they did not know the

function of try out; they did not know well how to see the validity and reliability

of their instrument either for test or questionnaire; they did not know how to

determine the appropriate and correct technique and formula of the validity and

reliability analysis of their instrument; and so on. So, in this paper the

writer needs to answer some questions; they are: 1) How are the knowledge of

students related to validity, reliability, and questionnaire? 2) How to analyze

the validity and reliability of questionnaire in appropriate way?

The objectives of this writing are to offer students

and researchers how to analyze validity and reliability of their questionnaire

as the instrument of research. The significance is that as reference for the

students and researchers so they are able to determine the appropriate and easy

technique and formula to analyze the validity and reliability of the

questionnaire.

Drost (2011) actually had offered generally about

technique and formula those could be used to see the extent of validity and

reliability, but there was no discussion about types of data as the basis of

technique and formula used. His discussion was in general and not specifically

mentioned about questionnaire matter.

In previous research and paper, actually the writer

had a research related to validity and reliability of instrument such as Sugianto (2011) that discussed about validity and reliability

of formative test, Sugianto (2016a) that discussed about validity and reliability

of English National Final Examination test, and also Sugianto (2017) that discussed about validity and reliability

of summative test. All the researches above are only all about instruments in

form of test for classroom matter. So there are no discussions related to

analysis for instrument of research matter. However, actually the procedures

are applicable for research matter as long as the form of the instrument is

also test like the test analyzed. Another previous writing is Sugianto (2016b) which discussed about the technique and

procedure of item analysis of objective test. This paper is type of item

analysis but special about objective test. The technique and procedure are

applicable for classroom matter test and research matter test/ instrument.

2.

Methodology

As written in the introduction

that the students still have problem in their research related to their

instrument, to help the students’ problem the writer conducted this

mini-research. The methodology in this mini research was conducted in order to

know the fact exactly their condition or knowledge especially about their

ability in analyzing the questionnaire in terms of validity and reliability. Therefore,

in collecting the data the writer used observation and interview as the

instrument. The subject of study is some students of English Department of

State Islamic Institute of Palangka Raya Indonesia who have written their proposal of thesis writing and have conducted

examination of their thesis. The data were

analyzed qualitatively.

After knowing the condition or knowledge of students

related to their ability and knowledge about instrumentation of research, the

writer would elaborate how to analyze the validity and reliability of the

questionnaire especially Likert scale questionnaire.

3.

Finding and Discussion

As stated in

introduction, actually the writer got the overview of students from the

writer’s experience became advisor of thesis writing. Then it was investigated

more by using procedure as stated in the methodology to find the real facts. From

the observation when the thesis examination and interview to some students, it

was found that although almost all of

them know what questionnaire is, but they did not know well how to see or

analyze the validity and reliability of the questionnaire; they did not know

the significance of analyzing validity and reliability of the questionnaire as

an instrument; as well as they did not know how to determine the appropriate

technique and formula of the validity and reliability analysis of the

questionnaire.

Based on the finding above it was needed to elaborate

the procedure, technique and formula to analyze the validity and reliability of

the questionnaire. The elaborations are in the following discussion.

3.1. Validity of

Questionnaire

The easiest procedure to analyze the validity of

questionnaire is based on items. Every item of questionnaire is analyzed to see

the validity of each item. The technique is by correlating the score of item to

the total score. So, it would use correlation formula. The popular one is

Product Moment correlation or also mentioned as Pearson Product Moment.

Determining Product Moment correlation is

actually based on the types of data. Actually there are some types of

correlation formula such as Pearson

correlation (Product Moment, Point-Biserial, etc.), Kendall rank correlation, and

Spearman rank

correlation. Types of data influence the types of

correlation formula that is appropriate to be used. Sugianto (2016b) wrote that the appropriate formula in analyzing the

data related closely to the types of data. For example if the data those would be correlated are continuous data/

variables (ordinal, interval, and ratio), the appropriate formula is Product

Moment correlation formula. But if one of the data is nominal (dichotomous),

Point-Biserial correlation formula is the best one (Brown, 2005). Hamzah & Abdullah (2011) also discussed about the concept of the relation about types of data to

the statistical measurement.

Especially for questionnaire, based on the types of data those would be

correlated which are continuous data, so it would be Product Moment correlation

formula. Product Moment formula can be calculate manually by following the formula and can also be

processed by computer software such as Microsoft Excel and SPPS. The formula is actually as follow:

Where:

N : number of pairs of

scores

:

sum of the products of paired scores

:

sum of X scores

:

sum of Y scores

:

sum of squared X scores

:

sum of squared Y scores

Interpretation of coefficient could consult to r-table. If the

coefficient r of the item is equal or more than the value of r-table (r ?

r-table), it means the item is valid. But if the coefficient r of

the item is less than the value of r-table (r < r-table), it means the item
is invalid.
3.2. Reliability of
Questionnaire
There are many formulas to analyze the
reliability such as Product Moment correlation, Flanagan, Rulon, Kuder-Richardson
(KR-20, KR-21), Hoyt, Spearman-Brown, and Cronbach Alpha. The use of them
depends on the technique and the type of data. So formulas, techniques, and types
of data are closely
related each other.
The easy and simple
technique to analyze the reliability of instrument is by single test – single
trial method. Single test – single trial method is a method where the test/
questionnaire is constructed one time (one test/ questionnaire) and would tried
out one times also. The method could be applied to the questionnaire. Then
since the types of data of questionnaire are continues data, so the appropriate
formula is Cronbach Alpha formula. Cronbach Alpha can be calculated manually by
following the formula and can also be processed by computer software such as
Microsoft Excel and SPPS. The formula is
actually as follow:
Where:
n :
number of items
: sum of variances of items
: total of variances
To calculate the variance (
, it can be used the
following formula:
Where:
N :
number of students
To interpret the Cronbach Alpha coefficient, although
still there are different opinion from experts related to the value of
reliability, but they gave caution related to their opinion. Panayides (2013) summarized experts' suggestions,
recommendations and caution related to the value of reliability and it can be
concluded that the reliability should be 0.70 or higher (r ? 0.70). For the
purpose that is really important 0.90 – 0.95 is better but in assumption that
the question contents are varied (not tend to duplication or paraphrasing).
4. Conclusion and Suggestion
4.1. Conclusion
Questionnaire
is popular to be the instrument of research. As the instrument of research,
questionnaire should also fulfill the characteristics of a good instrument such
valid and reliable. Valid and reliable instrument would be the qualification of
credible and good result of result or finding.
This mini-research found that the research subject did
not have adequate ability and knowledge to determine and process their
instrument especially in this case is questionnaire. The result showed that although almost all of them know what
questionnaire is, but they did not know well how to see the validity and reliability
of the questionnaire; they did not know the significance of validity and
reliability of the questionnaire as an instrument; as well as they did not know
how to determine the appropriate technique and formula of the validity and
reliability analysis of the questionnaire.
In order to help the
student to understand and to solve their problem related to the finding, it is
necessary to give how to analyze the validity and reliability of the
questionnaire. Simply based on the popular technique and the type of data, the
validity of questionnaire can be analyzed by using Product Moment formula, and
the reliability of the questionnaire can be analyzed by using Cronbach Alpha
formula. The calculation can be processed manually or by computer software such
as Microsoft Excel and SPSS.
4.2. Suggestion
There
are some suggestions related to this study:
1. Since
the writer delimited his research on questionnaire and found some problems of
students on it, so it will also be worth if other researches are also conducted
on other types of instruments such as tests. Test consists of varied forms such
as writing, speaking, Multiple Choice Question, essay and other types of
subjective or objective tests. These types of instruments are also potential
having the same problem.
2. If
the problems are found, it is better to give directly the problem solving such
as practical materials on your papers. Practical material means the materials
as you see on the discussion on this paper such as what the best technique of
validity and reliability of instruments is, how to determine the formula of
validity and reliability of the instrument, and types of data which influence
the decision in determining technique and formula.
3. On
this paper, the writer put the practical material related to the students'
problem although only general practical material. Therefore it is better if the
next paper would be written specifically the procedure, tabulation and example
of processing the data using the formulas offered so the students would be
easier to follow the instruction or material.
It is not a must to
calculate by manually. Nowadays, any activities in life are inseparable from
computer, and educational field is not exception. So, the analysis of validity
and reliability of instrument can also be processed by using computer software
such as Microsoft Excel and SPSS.