Student Assessment of LUCID

Student Assessment of LUCID, Spring 1998

In the final week of classes in the Spring of 1998, students using LUCID in two different workshops at Stony Brook were asked to assess the program. The version they were using included interactive models, considerable multilevel feedback, and networked reporting. The reporting feature was introduced late in the semester, however, and was not used successfully.

Each student was asked to agree or disagree with 10 positive statements regarding LUCID as well as provide open responses to three questions (see below). The graph below suggests positive student attitudes regarding the software.

wpe3E.jpg (27711 bytes)

The table below summarizes the positive statements made in the assessment along with the percent of students who agreed with these statements. As our administration tells us that it is very positive when freshmen are neutral, we also include the percent of students who did not disagree with the positive statements. A- Strongly Agree; B- Agree; C- Neutral

Statement	%A-B	%A-C
1. Computer activities are more enjoyable than text activities.	90	97
2. Computer activities help me learn more than text activities.	81	97
3. Computer models … help me understand concepts better than models I can’t interact with.	87	97
4. Instant feedback … increases the speed at which I can learn.	84	94
5. Instant feedback improves my skill in applying what I have learned.	90	97
6. Instant feedback increases my self-confidence in my learning.	84	97
7. Reporting answers to the class using the computer is better than reporting on the board.	48	87
8. I would recommend computer workshops over text workshops to a friend.	77	94
9. The computer activities would be helpful even without a workshop.	71	84
10. Working with others in a workshop with an instructor present helps me learn more than I would on my own.	84	94

Most Common Responses to Open Questions

1. What was the most valuable aspect of the computer activities?

53%-Instant feedback

16%-Interactive models

2. What was the greatest disadvantage of the computer activities?

22%-None noted

19%-Drawing was too time consuming. This primarily referred to the Lewis structure drawing tool used in assessed version. Based on these remarks, the tool was redesigned and is now significantly faster to use.

3. What was most in need of improvement?

19%-Nothing noted

16% -More explanations. These will not be included, as the aim of the software is to get students to generate these explanations themselves.

16%-Better recognition of answers. This primarily referred to the chemical reaction equations which had to be entered in a very specific fashion. Based on these remarks, algorithms were developed to analyze reaction equations so that they could be entered in a more flexible format.

Our Perspective on the Student Assessments

Question 7, regarding the benefits of networked reporting, is the lowest score (48% agreed). Networked reporting was introduced late in the semester and the first few attempts failed. Neither the students nor the instructor had developed the habits needed for effective use of the utility, and it was dropped until the coming semester. We view the low score for networked reporting as indicative of honest responses elsewhere.
The highest scores are for Questions 1 (90%), 3 (87%), and 5 (90%). Students responded most positively to the multilevel feedback and interactive models, finding work on the computer more enjoyable than work from a text.
Question 9 is low (71%) in comparison to Question 10 (84%).
While students see the software as having value in and of itself, they perceive the interactions with peers and instructors in the workshop as enhancing this value.