The Ala Wai Watershed Inquiry (AWI)
teacher-development workshop consisted of eight daylong meetings spanning the
period between March 1, 2001 and May 12, 2001. This workshop served as one in a
series of workshops designed to equip teachers with skills necessary to promote
inquiry and critical thinking in their classrooms. The main purpose of the AWI Workshop was to
strengthen teacher confidence and skills in standards-based teaching through
inquiry, problem solving, and critical thinking centered on the concepts of the
watershed and environmental stewardship.
Specifically, the AWI was
designed to address the concept of watershed stewardship in relationship to the
Ala Wai Canal, to assist teachers in understanding how to use the Hawaii State
Content Standards and Benchmarks in designing their curricula, to promote the
understanding and use of the inquiry process in science and other fields of
middle school education for addressing the content standards, to begin to
increase teachers’ understanding of how to develop critical thinking skills in
students, and to encourage the use of the scientific research investigation to
broaden the approach and scope of the current science curriculum used by
teachers.
In addition to the goals mentioned
above, the AWI Workshop was also designed
to strengthen those teacher communication skills necessary to
successfully engage in learning through inquiry. Teachers were required to
deliver a group oral presentation of their groups’ research investigations that
were conducted over the course of the workshop. They were also required to
submit a written research report of this investigation. Finally, teacher-participants were required
to develop and submit a written outline of a plan for an inquiry, standards-based
unit for future implementation in their classrooms. The workshop content was supported by the
expertise of an Environmental Education Specialist, a Resource Teacher, three
professional scientists (including two Environmental Biologists and a Neuroscientist/Science
Research Education Specialist), and two spokespersons for the Ala Wai Watershed
Association.
Teacher Demographics
Seven middle school teachers of
science, social studies, math, english, and physical education participated in
the workshop. Six of the teachers were
from
Assessment and Evaluation
Techniques for the Workshop
The seven teacher-participants were asked
to complete an attitude and knowledge Workshop Assessment prior to their
participating in the workshop (Pre-Workshop Assessment), during their
participation midway through the workshop (Day 4, Mid-Workshop Assessment), and
following their participation in the workshop (Post-Workshop Assessment).
Self-reported confidence items required teachers to place a vertical line on an
18 cm scale. These responses were then
measured and quantified for data analyses.
Teachers were also asked to write occasional “temperature checks” and a
“reflection” following each workshop. A
brief questionnaire was given to the teachers after they engaged in research
scenarios designed to provide them with experience and practice to select and
use the correct research tools for testing particular hypotheses about
AWI Program Evaluation
Methods
Teacher reflection and “temperature
check” comments were placed into categories based on whether they addressed the
workshop in general; Ala Wai Watershed, watersheds in general, and
environmental stewardship; scientific inquiry and the research investigation
process; or standards-based education, including content standards and the
standards-based unit. (Note that all
teacher comments are quoted exactly as expressed, but may include additions by
the evaluator, indicated with brackets, for clarity.) The comments were then used, where appropriate,
to support and clarify quantitative data from the Pre-, Mid-, and Post-Workshop
Assessments, Program and Teacher Expectation Surveys, and PD-Credit
Evaluations.
The data from the items in the
Workshop Assessments (Pre-, Mid, and Post-Workshop Assessment), were summarized
and graphed. Where appropriate, one-way
ANOVAs with repeated measures were used to determine differences between the
pre-, mid-, and post-workshop data. When the results were statistically significant,
Neuman-Keuls multiple comparison tests were used to determine differences
between pairs of means. The Teacher and
Program Expectation Surveys used Likert-scale items requiring the
teacher-participants to decide whether various expectations were not at all
addressed, somewhat addressed, or completely addressed in the workshop. The
data from these surveys were summarized and graphed as percent to determine the
extent to which teachers felt that the expectations were met. Items from the research scenario
questionnaires were analyzed using the dependent t-test. The PD-Credit
Evaluation consisted of 10 elements of Quality Professional Development
arranged as a Likert-like scale. The evaluation scale for each element ranged
from 1=does not meet the standard, 2=partly meets the standard, 3=meets the
standard, 4=more than meets the standard, to 5=meets the standard to a high
degree. Teacher participant data from the PD-Credit Evaluation were summarized
using the mean and range for each item.
Evaluation Overview and
Summary of Findings
This evaluation focuses on the
impact of the AWI workshop on enhancement of teacher confidence and knowledge
in engaging students in standards-based learning about environmental
stewardship through scientific inquiry.
To this end, the evaluation specifically examines teacher-participant
understanding of, and confidence in teaching about and/or through, the concepts
of watershed and environmental stewardship, scientific inquiry and the research
investigation process, and content standards and standards-based units.
General comments gathered from
teacher reflections and “temperature checks,” as well as quantitative data from
assessments, surveys, and evaluations indicate that the workshop engaged
teachers in a successful professional development experience.
The major findings in this evaluation are that participation in the AWI
Workshop resulted in overall increases in both teacher knowledge about, and
teacher confidence to use in their teaching 1) the Ala Wai Watershed and
environmental stewardship, 2) scientific inquiry and the research investigation
process, and 3) content standands and standards-based units. General teacher-participant reflection
comments made at the end of the workshop support the quantitative data obtained
from assessments and surveys, indicating a gain in knowledge in the three focal
areas of the workshop (see Tables 1-3 below).
Table
1. Teacher-participant general
statements about the Ala Wai Watershed and environmental stewardship.
|
“This workshop was
completely successful at addressing the speakers, provided background
information, and involved us in doing field studies and our own research study on the |
|
|
|
“Learning about the |
|
“For so many years I drove past the |
Table
2. Teacher-participant general
statements about Scientific Inquiry and the Research Investigation Process.
|
“My knowledge/understanding of research design is refined
because of this workshop.” - Teacher #1 |
|
|
|
“I will definitely
be able to use the research process in all of my classes.” - Teacher #2 |
|
|
|
“I now know how to
use the research investigation process.” -
Teacher #3 |
|
“Now I have some
ideas of how I can use scientific inquiry in my math, language arts, and
health classes. I am excited
about teaching scientific inquiry.” |
|
“I understand the research process better than I did in
college! |
|
|
|
“This workshop did
well in helping the teachers to learn and to understand each step of the
process in doing [a]
research investigation.” - Teacher #4 |
|
|
|
“I became
acquainted with some of the tools scientists use in their investigations and
the sequence of steps within the research investigation
process.” - Teacher #6 |
|
|
|
“I am grateful for
this workshop for it has challenged me to expand my knowledge of inquiry
process and the skills
needed... .” - Teacher #7 |
|
“Research Investigation provides the teacher with an
awesome instructional tool to help implement
standards-based learning and support the emphasis on what is learned
rather than what is taught.” |
Table
3. Teacher-participant general statements about Standards and the Standards-
Based Unit.
|
“I finally understand what content standards, benchmarks,
performance indicators, and performance standards
are! It took this program to finally
make sense of all of these terms.” - Teacher
#4 |
|
|
|
“I gained the following as a teacher from this workshop:
Looking at standards and attempting to write performance
indicators.” - Teacher #5 |
|
|
|
“This area [of the workshop] really helped to relate the
scientific process and the standards in my mind.” - Teacher #7 |
|
“I now fully understand and appreciate how engaging
students in research can address a number of the HCPS
II... .” |
As teacher knowledge increased, so
did overall teacher-participant confidence in the areas covered in the
workshop, as indicated by the teachers’ self-reported confidence level in the
accurateness of their responses on the Workshop Assessment questions (Figure
1). By the end of the workshop, general
teacher confidence in understanding and being able to apply the concepts
covered in the workshop more than doubled to a level above “confident,” which
was a statistically significant increase over the substantially below
“confident” level exhibited by teachers prior to participating in the workshop.
Mean (+SEM)
Confidence (measured in cm) Completely Confident
Confident Not at all Confident Time of Assessment
Figure 1. Teachers’ self-reported confidence in their answering all of the items on the Workshop Assessment accurately before (Pre), during (Mid), and after (Post) their participation in the workshop. Confidence ratings were made by teachers after they completed the items on the assessment. A one-way repeated measures ANOVA indicated that teacher confidence in the accuracy of their responses to items on the assessment increased significantly by the end of the workshop (F2,12= 6.43, p < 0.02). This indicated that teachers were becoming more comfortable with the concepts covered in the workshop.
*significantly
different from Pre-Workshop (p <
0.05)
Teacher
comments from reflections at the end of the workshop described this
post-workshop increase in confidence: “In the area of addressing standards in
the AWI program..., I personally feel a lot more confident about implementing
it in my class; or, to be more specific, ‘I get it now!’” and “It [the workshop] has helped me feel a lot
more confident about research investigation... .”
A general goal of the AWI
Workshop was to instill teacher confidence in the use of the concept of
environmental stewardship applied to the
Responses from an overwhelming
majority of the teachers indicated that the teacher expectation centering on
learning and understanding watersheds was completely addressed (Figure 2). Teacher #4 reflected, “I understand what the
Ala Wai Watershed is and what a watershed is.”
However, one teacher (Teacher #6) indicated that this component of the
workshop was not at all addressed (Figure 2).
This teacher reflected, “What did this class teach us about a
watershed? What did you want us to learn
about a watershed? I have no idea.”
Figure 2. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop addressed the expectation for teachers to learn about and understand watersheds.
Five of seven of the workshop
participants responded that the teacher expectation of implementing and
observing a model of a watershed project was fully addressed, while one teacher
felt that it was somewhat addressed (Figure 3).
Teacher #7 reflected, “I feel that the AWI Program did address this
portion of the [teacher] expectations.”
However, similar to how she felt about the previous teacher expectation,
Teacher #6 responded that this expectation also was not at all addressed.
Figure 3. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop addressed the expectation for teachers to implement and observe a model of a watershed project in action.
As can be seen in Figures 4-6 below,
there was unanimous agreement that the three project expectations related to
the Ala Wai Watershed--conducting a research investigation, presenting an oral
report and producing a written report-- were completely addressed in the
workshop.
Figure
4. Teacher-participants’ responses (shown as percent) regarding the extent to
which the AWI Workshop addressed the expectation for teachers to conduct a
research study (with physical and social components), from beginning to end,
that centers on the Ala Wai.
Figure 5. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop addressed the expectation of teachers conducting an oral report on their research investigation of the Ala Wai.
Figure 6. Teacher-participants’ responses (shown as percent)
regarding the extent to which the AWI Workshop addressed the expectation of
teachers writing a research report on their research investigation of the Ala
Wai.
Teachers’ knowledge about
watersheds and environmental stewardship were assessed in the Workshop
Assessment (Figures 7 and 8). As shown
in Figure 7, participation in the workshop resulted in a dramatic 70% increase
in teachers able to demonstrate a good understanding and a decrease from 90 to
30% of the teachers having only a poor understanding of the term “watershed.”
Figure 7. Percent of 7 teachers having a good, fair, and poor understanding of the term “watershed” before (Pre-), during (Mid-), and after completing (Post-) the workshop.
One teacher-participant (Teacher #7) reflected on her Ala Wai Watershed learning experience with, “Learning about the Ala Wai Watershed was quite eye-opening... .”
In contrast, the AWI Workshop did not
appear to have a major influence on teacher understanding of environmental
stewardship (Figure 8). By the end of
the workshop, only one teacher (Teacher #7) increased her understanding of this
concept (from poor to good). At the end
of the workshop, Teacher #7 reflects her new knowledge about the Ala Wai
Watershed in a statement emphasizing stewardship: “For so many years, I drive
past the AW and never knew its needs and dilemma.”
Figure 8. Percent of 7 teachers having a good, fair, and poor understanding of the term “environmental stewardship” before (Pre-), during (Mid-), and after completing (Post-) the workshop.
Although teachers’ understanding of
environmental stewardship did not appear to be greatly influenced by their participation in the
AWI Workshop, teacher confidence in the ability to teach and engage students in
environmental stewardship was dramatically changed. Figure 9 shows the mean teacher confidence in
ability to teach and engage students in environmental stewardship before,
during, and after participating in the workshop. Teacher confidence level
increased significantly from near not at all confident before the workshop
(mean + SEM = 2.63 + 0.71) to confident by the end of the
workshop (7.43 + 1.92) [Figure 9].
Mean (+ SEM)
Confidence (measured in cm) Completely
Confident Time of Assessment Not at all Confident Confident
Figure 9. Teachers’ self-reported confidence in their ability to teach and engage their students in environmental stewardship before (Pre), during (Mid), and after (Post) their participation in the workshop. A one-way repeated measures ANOVA indicated that participation in the workshop significantly influenced teacher confidence in their ability to teach and engage their students in environmental stewardship by the end of the workshop (F2,12 = 4.22, p = 0.04).
*significantly
different from Pre-Workshop (p <
0.05)
Scientific Inquiry and the
Scientific Research Investigation (RIP) Process
One primary goal of this workshop
was to assist teachers in learning about and developing skills in the research
investigation process so that teachers could begin using scientific inquiry in
standards-based science teaching.
Specifically, it was expected that teacher-participants would become
familiar with and be able to begin to guide their students in the steps
necessary to conduct scientific research to gain an understanding of the world
around them. It was also expected that teachers would gain an appreciation for how
the inquiry process used in science can be generalized to other academic
courses. Increasing teacher comfort
level in emphasizing the use of statistics in data analyses and decision-making
by their students was a primary goal of the research investigation/inquiry
content of the workshop.
The teacher-participants unanimously
indicated on the Program and Teacher Expectation Surveys that all workshop
expectations related to scientific inquiry and the research investigation
process were at least somewhat addressed, with a majority of teachers
indicating that five of these six expectations were fully addressed (see
Figures 10-14 below).
All seven teachers responded on the
Program Expectation Survey that they had engaged in research investigations
which generated new knowledge (Figure 10).
Figure 10. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop engaged them in research investigations which generate new knowledge.
The teachers were also unanimous that
the program expectation for teachers to understand how the parts of a research
investigation relate to each other was fully addressed in the workshop (Figure
11). For example, Teacher #5 stated, “I
have learned the step-by-step process of research investigation. I now know how each part is connected;” and
Teacher #6 claimed, “Our teachers did a good job in teaching us the
interrelationship between the parts of a research investigation.”
Figure 11. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop helped them to understand the interrelationship between the parts of a research investigation (introduction, methods, results, discussion and conclusion, and action plan).
The majority of the participants
expressed that the program expectation of understanding the interrelationship
between research design and data analysis was completely covered in the
workshop; however, three of the seven teachers also felt that this expectation
was only partially addressed (Figure 12).
Figure 12. Teacher-participants’ responses (shown as
percent) regarding the extent to which the AWI Workshop addressed the
understanding of the interrelationship between research design and data analysis.
All participants felt that the strengthening of communication skills necessary for researchers was addressed in the workshop, with three out of seven indicating that this program expectation was addressed completely (Figure 13).
Figure 13. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop addressed the strengthening of all communication skills that are necessary for researchers.
Five of the seven
teacher-participant responses on the Teacher Expectation Survey indicated that
the learning of statistics for use in the scientific inquiry process was
completely addressed in the workshop (Figure 14). Teacher #1 claimed, “Standard deviation
[is]...among the most valued things that I gained;” and Teacher # 7 reflected,
“Regarding the area on statistics, I truly learned something new here...” and
“I [had] often wondered how scientists came to the conclusions they came to
using data.” One teacher (Teacher #7)
stated, “Although understanding the null hypothesis is simple, it was confusing
for my brain and I found it a fun and challenging concept to grasp.”
Figure 14. Teacher-participants’ responses (shown as
percent) regarding the extent to which the AWI Workshop engaged them in the
learning of statistics.
The Majority of the teachers also felt that their
expectation for learning about specific statistical tests and directional and
non-directional hypotheses (one- and two-tailed tests) was completely addressed
(Figure 15). One teacher reflected, “How
thrilling to learn [about]something like the ‘null’ [hypothesis].”
Figure 15. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop engaged them in learning about correlation and t-tests and using different tailed tests.
The percent of teachers
demonstrating a good understanding of the role statistics play in the research
process increased from 0 % to 60 % by the end of the workshop, with a
corresponding six-fold decrease in the percent of teachers with poor
understanding (Figure 16).
Figure 16. Percent of 7 teachers having a good, fair, and poor understanding of the role statistics play in research before (Pre-), during (Mid-), and after completing (Post-) the workshop.
For decision-making based on data
collected in the research investigation, the AWI Workshop emphasized graphing
and interpretation of experimental and correlational data, covering the bar
graph (histogram), line graph, and scatterplot.
The Workshop Assessment examined the impact of the workshop on the
teachers’ ability to describe, understand and interpret graphs for making
decisions based on research investigation data.
The assessment data indicate that teacher understanding and
interpretation of the graphs increased substantially over the course of the AWI
Workshop. For example, as shown below
in Figures 17-19, teacher-participants’ abilities to describe and interpret bar
graphs were dramatically improved over the course of the workshop. The percent
of teachers able to provide excellent descriptions of the data presented in a
bar graph increased from zero to sixty percent, while poor descriptions
decreased from 70% (Pre-Workshop) to 0 % (Post-Workshop) [Figure 17].
Figure 17. Percent excellent, good, and poor responses for 7 teachers describing a bar graph (histogram) comparing plant growth under different fertilizer conditions before (Pre-), during (Mid-), and after completing (Post-) the workshop.
By
the end of the workshop, teachers also exhibited a large increase (0 %
Pre-Workshop to 55 % Post-Workshop) in ability to correctly interpret the data
presented in a bar graph (Figure 18).
This large percent increase in
interpretation ability corresponded with a dramatic percent increase (from 0 % to
70 %) in understanding of the concept of error in decision making and a dramatic
decrease (from 100 % Pre-Workshop to 10 % Post-Workshop) in lack of
understanding of error (Figure 19).
Figure 18. Percent Correct and incorrect responses for 7 teachers interpreting data presented in a bar graph (histogram) comparing plant growth under different fertilizer conditions speed before (Pre-), during (Mid-), and after completing (Post-) the workshop.
Figure 19. Percent excellent, good, and poor responses for 7 teachers recognizing the role (importance) of variability (error bars) in interpreting the data presented in a bar graph (histogram) comparing plant growth under different fertilizer conditions speed before (Pre-), during (Mid-), and after completing (Post-) the workshop.
Figure 20 shows the mean confidence
of the teachers in their ability to success-fully engage students in research
in their subject area before, during, and after their participation in the AWI
workshop. As can be seen in the figure, prior to the workshop, teachers rated
themselves as being slightly less than confident (mean + SEM = 7.01 +
1.11 compared to 8.25), while after their participation in the workshop, they
felt slightly more than confident (9.24 + 1.43 compared to 8.25). A repeated measures ANOVA indicated that mean
teacher confidence for engaging their students in research did not differ
across the three workshop periods.
However, because of the small sample size (n=7) and relatively small variation among the means,
the ability for the ANOVA to detect a significant workshop effect on teacher
confidence was very weak. Thus, the negative statistical finding on workshop
influence on teacher confidence level in successfully engaging students in
research should be cautiously interpreted. (See Figure 20 caption for further
explanation of the power issue.)
Not at all Confident Confident Completely
Confident Mean (+ SEM)
Confidence (measured in cm)
Time of Assessment
Figure 20. Teachers’
self-reported confidence in their ability to successfully engage students in
research in their subject area before (Pre),
during (Mid), and after (Post) their participation in the workshop. A one-way
repeated measures ANOVA indicated that the teachers’ confidence was not significantly
different across the three workshop assessment periods (F 2,12 = 1.75, p
> 0.05). Due to the small sample size
and relatively small variation among the means, the power of this test to detect
a significant workshop effect on teacher confidence was very low (power of
performed test with alpha = 0.05: 0.14) compared to the desired power of
0.80. Thus, the negative statistical finding
should be interpreted cautiously (see text for additional discussion).
Figure 21 below presents confidence
values plotted for each teacher-participant.
At the end of the workshop, four of the seven teacher-participants
showed clear, steep increases and two (Teachers #1 and #5) showed very slight
increases in confidence in this ability compared to their confidence before
participating in the workshop. In
contrast, one teacher (Teacher #6) showed a drop in confidence, from confident
to somewhat less than confident, between the Pre- and Post-Workshop assessments
(see Figure 21).
Figure 21. Individual teacher (T) Pre- and Post-Workshop
confidence level in ability to successfully engage students in research in
their subject area.
Teacher #7, whose confidence in her ability to
successfully engage students in research increased from less than confident to
confident (see Figure 21 above), actually engaged her students in the
scientific inquiry process that she was learning while the workshop was still
in progress. She described her
experiences with her students with, “You
know it amazed me that when given the opportunity students came up with ‘better
research’ questions than I could think of,” and “The students then used the
‘If...then...’ form to come up with a hypothesis. With each hypothesis they generated, we went
through the criteria list- testable, materials, subjects available and stated
as an answer to a question- and I believe that the students felt successful and
at ease with this.”
According to their Workshop
Assessment responses, teachers revised and refined their understanding of “research” over the course of the workshop.
Although the percentage of teachers who had a poor understanding of the meaning
of research, as it pertained to what was covered in the workshop, did not
change by the end of the workshop, the percentage of teachers with a fair
understanding decreased six-fold and those with a good understanding increased
by fifteen percent (see Figure 22).
Figure 22. Percent of 7 teachers having a good, fair, and poor understanding of the word “research” before (Pre-), during (Mid-), and after completing (Post-) the workshop.
A variety of strategies and tools were used in the
AWI Workshop to enhance teacher learning of the research investigation process
and to facilitate the development of doable lessons. Specifically, sample inquiry lessons, a
research investigation flowchart developed by the workshop instructors,
storyboards, practice research scenarios, and a statistics manual written by
the Science Research Education Specialist were used to teach the process of
scientific inquiry and its application to the Ala Wai Canal Watershed. The impact of these teaching strategies on
learning was assessed through the evaluation of the Teacher Expectations Survey
and teacher reflection statements. The
success of the practice research scenarios as a strategy was also determined
through analysis of teacher responses on a brief questionnaire.
The teacher expectations for teacher
strategies encompassed six of the items on the Teacher Expectations Survey (see
Figures 23-27 below). Almost three
quarters of the teachers felt that the workshop partially addressed their
expectation for providing them with skills to create doable units (Figure
23). In addition, one teacher responded
that this teacher expectation was completely addressed, while another responded
that it was not at all addressed.
Figure 23.
Teacher-participants’ responses (shown as percent) regarding the extent
to which the AWI Workshop addressed the expectation for teachers to be able to
create doable lessons/units.
All but of one of the workshop
participants also felt that the expectation for learning classroom-friendly inquiry activities
was at least partially addressed, with one teacher responding that they were
completely addressed (Figure 24). Teacher #6 stated, “The observation and pulse rate activities
were very good in teaching the scientific inquiry process.” “These....two activities....were classroom
friendly.” One teacher, however, indicated
that this expectation was not at all addressed.
Figure 24. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop addressed the expectation for teachers to learn inquiry activities that are classroom friendly.
Six of the seven teacher-participants
responded that the expectation for them to learn more hands-on inquiry-based
lessons was somewhat addressed (Figure 25). Teacher #3 stated that “The
research investigation flowchart was a big help.” “[The research investigation]
flowchart allowed us to go through the [research investigation] process-hands
on,” and Teacher #7 stated that “the
research investigation flowchart combines ‘hands-on’ activities and ‘thinking’
activities quite well.” In contrast,
one teacher indicated that this expectation was not at all addressed (Figure
25).
One teacher (Teacher #7) was
actually putting science inquiry techniques she was learning in the workshop
into practice in her classroom. She
reflected, “You know it amazed me that when given the opportunity students came
up with ‘better research’ questions than I could think of,” and “The students
then used the “If...then...form to come up with a hypothesis. With each
hypothesis they generated, we went through the criteria
list...testable...materials and subjects available...stated as an answer to
research question and I believe the students felt successful and at ease with
this.”
Figure 25. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop addressed the expectation for teachers to be able to learn more “hands-on” activities to take back and use with their students.
One set of activities within the AWI Workshop were four practice
research scenarios involving contrived observations made on the Ala Wai
Watershed accompanied by real and laboratory-doctored water samples. These
scenarios were designed so that each would suggest a potential water quality
problem that could be studied in the laboratory using available research instruments
such as conductivity meters, hydrometers, pH indicators, and kits for measuring
dissolved oxygen, copper, nitrate, nitrite, phosphate and ammonia. The purpose of these scenarios was to engage
teachers in activities that would force them to begin to problem solve issues
surrounding a watershed using critical thinking and the research investigation
process. Post-scenario
questionnaires administered after
completion of the first and third scenarios were used to assess whether
teachers’ confidence in guiding themselves through the water quality research
investigation process was influenced by this activity.
Figures 26 and 27 present
self-reported confidence values for teachers following their participation in
the first and then the third research scenarios. As can be seen in Figure 26, practice with
the research scenarios did not affect teacher confidence regarding their
ability to successfully conduct research in water quality issues. Similarly, teacher confidence in their
ability to explain to others the research process they used was unchanged
(Figure 27).
Not at all Confident Confident Mean (+SEM)
Confidence (measured in cm) Completely Confident n=7 n=7 Post Scenario #3 Post Scenario #1 Confident n=7 n=7 Post Scenario #3 Post Scenario #1
Figure 26. Teachers’ self-reported confidence in their ability to successfully engage in water quality testing research following participation in one and three practice research scenarios. A dependent t-test indicated that there was no difference in mean teacher confidence measured by the two research scenario questionnaires (tdf =6 = -0.27, p = 0.80).
Comments from teachers revealed that they felt that not enough time was
given for them to fully benefit from this practice activity. As discussed at the end of this evaluation, a
teacher strike necessitated the abbreviation of time permitted for certain
activities such as these. Teacher #3
said, “Some of the meetings seemed very rushed (data collection/practice [four
research scenarios]),” and Teacher #6 stated, “I felt cheated when our time was
cut short on the day we were to do four labs [research scenarios]. I would have liked to experience using all of
the tools available... .”
Mean (+SEM)
Confidence (measured in cm) Not at all Confident Completely Confident Post Scenario #3 Post Scenario #1 n=7 n=7 Confident
Figure 27. Teachers’ self-reported confidence in their ability to explain the water quality testing research process to another person following participation in one and three practice research scenarios. A dependent t-test indicated that there was no difference in mean teacher confidence measured by the two research scenario questionnaires (tdf =6 = -1.23, p = 0.26).
All of the participants indicated
that their expectation for the workshop to enable them to be able to use
activities/ideas to teach scientific inquiry was at least partially addressed,
with almost three-quarters of them indicating that this expectation was
completely addressed (Figure 28).
Figure 28.
Teacher-participants’ responses (shown as percent) regarding the extent
to which the AWI Workshop addressed the expectation for teachers to be able to
use activities/ideas to teach scientific inquiry.
Most
of the teachers, in their reflections and temperature checks, mentioned the
value of the Research Investigation Storyboard and the Research Investigation
Flowchart for teaching scientific inquiry.
Some comments were: “The Research Investigation Flowchart will encourage
students when they visually see how organized and sequential[ly] the Research
Investigation Process flows along,” “The techniques shown in the chart provide
a map for learners to see what successful learning looks like,” [using the flowchart to] “go over the parts
of the research investigation process step-by-step helped me to understand what
I was looking for. The flowchart is very
helpful and can be adjusted to fit most types of research.” Teacher #6 expressed, “The Research
Investigation Flowchart is a mode of delivery I can use to teach any content in
science.” It is a tool I can use to meet
all the scientific inquiry skills in the science content standards.” “I fully understand and appreciate how the
Research Investigation Flowchart can cover more than half of the science
standards in HCPS II.”
Teacher-participants also
consistently reflected positively about the value of the storyboard for the
planning of student research investigations.
For example, Teacher #4 commented, “Because we did a complete storyboard
for both the physical and social investigations, the data collection/analysis
portion of our investigation was a piece of cake,” and Teacher #6 explained the
value of using the storyboard: “The storyboard is very important in helping our
students to design and conduct a research investigation study because it helps
to clarify and elaborate step-by-step the process of the investigation. With
the scrutinizing eyes of peers and teachers, this will eventually save them
from making unforseen mistakes, not to mention the waste of time and costly
supplies and materials.” Finally,
Teacher #7 stated, “I love the idea of the storyboard. I think this is an
excellent way to help me access where the students are at, offer assistance to
those who are not progressing, and to give encouragement to those who are
progressing.”
Teachers were almost equally split
as to how much the workshop addressed their expectation to be able to use the
research investigation inquiry process to devise and implement a School-Wide
6-8 grade Theme Connection (Scope and Sequence) [Figure 29]. However, they all indicated that their
Figure 29. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop addressed the expectation for teachers to be able to be able to use the research investigation inquiry process in order to devise and implement a School-Wide 6-8 Theme Connection (Scope and Sequence).
expectation
of being able to develop project-based and interdisciplinary units was
addressed, with one-third responding that it was completely addressed (Figure
30).
Figure 30. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop addressed the expectation for teachers to be able to develop project-based and interdisciplinary units. (One workshop participant did not respond to this item.)
The seven AWI teachers unanimously agreed that the
standard for the five Elements of Quality Professional Development relating to
inquiry and actively engaging workshop participants in research were either met
or exceeded. All seven teacher
participants agreed that the workshop content at least met the standard for an
on-going and sustained process (PD-Credit Evaluation: mean = 4.57, range = 2.0), with all but one
teacher claiming that the standard was more than met or met to a high degree.
Teachers were also in overwhelming agreement that the standard for active
engagement was exceeded (PD-Credit Evaluation:
mean = 4.0, range = 2.0). This
element includes modeling what needs to occur in the classroom, inquiry-based
instruction/learning, and the modeling of practices by the facilitator. Teacher #7 stated “Research investigation embraces
goals that are critical for a successful learning environment.” The practice of
the scientific research process is commonly a collegial effort and, and
according to the teachers, this standard was met or exceeded in the workshop
(PD-Credit Evaluation: mean = 4.57,
range = 2.0). The teachers also felt
that the use of the inquiry approach, using the research investigation process
to learn science, as demonstrated in the workshop, focused on the element of
result-oriented instruction and student learning (PD-Credit Evaluation: mean = 4.43, range = 1.0). For example, Teacher #7 stated “Research
investigation...provides opportunity for individual self-direction, provides
opportunity for each individual to perform well and to take pride in
learning...and provides out-of-school learning experiences.” Finally, six of the seven
teacher-participants expressed their view that the workshop met the reflection
standard (mean = 4.14, range = 3.0). The research investigation process itself
involves continual reflection, analysis, refinement, and improvement.
Instruction of the research investigation process to the teacher-participants
was challenging, enhanced learning, and connected with practice.
Standards/Standards-Based
Unit
A major goal of this
workshop was for teachers to become more comfortable with the use of standards
in their classroom instruction.
Specifically, through the workshop, it was hoped that teachers would
develop a greater understanding of content standards, make connections between
content standards and classroom activities, and learn about and learn how to
develop a standards-based unit plan.
According to the results of the
Teacher Expectation Survey, the majority of teacher-participants felt that all
four of their expectations regarding the standards component of the AWI
Workshop were at least somewhat addressed (see Figures 31-34 below). The seven
workshop participants felt that the meaning of
“standards” was either partially
or fully addressed (Figure 31).
Figure 31. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop addressed what “standards” really mean.
Two-thirds of the teachers felt that
their expectation for reflecting on content standards and making connections
between them and the classroom was somewhat covered in the workshop, while
one-third felt it was not at all addressed (Figure 32).
Figure 32. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop engaged them in reflecting on content standards, and learning how to make connections between standards and classroom activities. (One of the seven workshop participants did not respond to this item on the survey.)
The majority (85%) of the workshop
participants indicated that their expectation of achieving a better
understanding of content standards (Figure 33) was at least somewhat addressed,
with two teachers feeling that it was completely addressed. In contrast, one
teacher (Teacher #2) responded that this expectation was not at all addressed
in the workshop. However, in her reflection on content standards, she stated,
“Although we were unable to cover much on content standards, I feel that enough
information was disbursed to give me a better understanding of what is
expected.”
Figure 33. Teacher-participants’ responses (shown as percent) regarding the extent to which the AWI Workshop helped them to gain a better understanding of content standards.
Slightly more than half of the
participants indicated that the teacher expectation of learning about, and how
to develop, a standards-based unit was at least somewhat addressed, with one
teacher indicating that she thought that these aspects of the standards-based
unit were completely addressed in the workshop (Figure 34). However, this is in contrast to 43% of the
teachers who felt that this component
was not at all addressed in the workshop.
One reason for teachers feeling that this element of the standards
component of the workshop was not addressed was expressed in a reflection from
Teacher #3: “I am still not clear on how
to develop a standards-based unit plan incorporating research investigation
based on the Social Studies standards because the SS standards were not
addressed.”
Figure 34. Teacher-participants’ responses (shown as percent) regarding the extent to which learning about the standards-based unit plan, and how to develop one, were addressed in the AWI Workshop.
Based on their responses on the AWI
Program Expectation Survey, all seven of the workshop participants felt that
the program expectations regarding standards were at least somewhat addressed
(see Figures 35 and 36 below). Six of the seven teachers responded that the
workshop completely addressed teachers’ understanding and appreciation for how
engaging students in research can address a number of the Hawaii Content and
Performance Standards (HCPS II) [Figure 35].
One teacher-participant (Teacher #7) stated that “this area really
helped to relate the scientific process and the standards in my mind.” However, only two of the seven teachers felt
that the workshop completely addressed teachers knowing, understanding, and
implementing HCPS II in their class (Figure 36).
Figure 35. Teacher Participants’ responses (shown as percent) regarding the extent to which the AWI Workshop helped teachers to understand and appreciate how engaging students in research can address a number of HCPS II.
Figure 36. Teacher Participants’ responses (shown as percent) regarding the extent to which the AWI Workshop helped teachers to know, understand, and implement HCPS II in their class.
Prior to taking the workshop, teachers reported that they were not
confident in addressing standards in
their classroom (see Figure 37 below). However, the teachers’ confidence in
their ability to address content standards in their classrooms significantly
increased by the end of the workshop, compared to their confidence before and
midway through the workshop.
Mean (+ SEM)
Confidence (measured in cm) Confident Completely Confident Not at all Confident
Time of Assessment
Figure 37. Teachers’ self-reported confidence in their ability to address content standards in their classroom before (Pre), during (Mid), and after (Post) their participation in the workshop. A one-way repeated measures ANOVA indicated that participation in the workshop significantly influenced teacher confidence in their ability to address content standards in their classrooms by the end of the workshop (F2,12= 5.51, p = 0.02).
*significantly different from Pre-Workshop (p < 0.05) **significantly different from Mid- Workshop (p < 0.05).
All seven teacher participants felt
that the standard was met for focus on the Hawaii Content and Performance
Standards (PD Credit Evaluation: mean = 4.14, range = 2), with five of them
expressing that the standard was exceeded.
According to one teacher’s reflection, “[The Research Investigation
Process] ties into science standards,.” and
“I feel confident that I can address certain standards with research
investigation.” However, three of the
seven participants expressed dissatisfaction with coverage of material related
to development of a standards-based unit plan. For example, Teacher #5 stated, “I think we should have
taken more time to learn how to write standards-based units and lessons,” while Teacher #6 said she “would like to see
what a standards-based unit plan looks like” and “...we were never taught how
to put a plan together.” Finally, one
participant (Teacher #4) reflected, “I ...am still not 100% clear about
performance indicators.”
Additional Evaluation
Findings and Conclusions
In addition to the quantitative data
and specific comments supporting the interpretation of a positive professional
learning outcome from the AWI Workshop, general teacher reflection and
“temperature check” comments made throughout the workshop illustrate the
overall positive impact of the workshop on teacher-participants (Table 4).
Table
4. General AWI Workshop
teacher-participant comments.
|
“This workshop did well in helping the teachers to learn
and to understand each step of the process in doing [a]
research investigation.” - Teacher #2 |
|
|
|
I have learned a lot from this class [workshop] and
enjoyed myself as well.” - Teacher #4 |
|
|
|
“It took another 3-credit class on standards and $270.
for me to appreciate this workshop and what it can offer to
help me deliver the standards to my students.” - Teacher #6 |
|
“The inquiry process we learned was excellent... .” |
|
“I definitely saw the treasure of the Research
Investigation Process and how it could be applied to anything
taught in the classroom.” |
|
|
|
“This was a most rewarding workshop and I’m glad I stuck
with it. Best of all, it [the workshop] makes the standards
so much more user-friendly.” - Teacher
#7 |
A number of teacher reflection
statements also made it clear that teachers found this workshop to be both
interesting and challenging. For
example, Teacher #1 stated that the workshop provided “high variety and
challenging activities;” Teacher #4
claimed, “Sometimes activities/concepts are challenging, but in a good
way. Exercising my brain feels great!;” and
Teacher #7 commented, “Very, very challenging.
You have forced me to use parts of my brain that [were] anorexic.”
Finally, the teacher-participants
attributed the success of the workshop, in part, to the skills and attitudes of
the instructors and facilitators (Table 5).
Table
5. AWI Workshop teacher-participant
comments relating to workshop staff.
|
“Knowledgeable and caring instructors.” - Teacher #1 |
|
|
|
“One of the
most predominant [strengths] is the many ‘experts’ that are brought in to explain
concepts. Because they are experts in their
fields, we get the most detailed and thorough information available; and it is presented by people
who are truly enthusiastic about what they are presenting. As a teacher,[I feel] just the enthusiasm
alone is enough to catch an audiences attention. I also feel that the coordinators for the workshop
have gone above and beyond to make sure that there are many opportunities for interesting and innovative things to happen.” - Teacher #2 |
|
|
|
“Instructors are
knowledgeable and really want to help us understand material. Activities are varied and interesting.” - Teacher #4 |
|
|
|
“You [the workshop instructors] all worked well as a team.” - Teacher #7 |
The AWI Workshop was conducted during
what might be considered a “difficult” period of the 2001 academic year. The
teachers were involved in a union-based job action, which included the planning
and conducting of a teacher strike. The planning stages of the strike were very
stressful, emotionally-draining, and time-consuming for the AWI
teacher-participants. The strike itself had a major impact on the planning, and
actual scheduling and completion of the workshop. During the period of time including April 5
through the first week of May, the teachers were out on-strike and previously
scheduled workshop days could not be held.
In addition, teachers had to play catch-up with their students at the
end of the strike and could not attend in-service activities. The teachers
expressed a high level of post-strike stress, which also interfered with
workshop progression. As a result of the
strike-caused disruption, six of the workshop days were crammed into a single
month, and the originally-planned 10-day workshop was forced into eight days.
Thus, the depth and breadth of a number of the previously planned workshop
elements had to be abbreviated. One area
that was particularly affected by this was the content standards and the
standards-based unit. Also, the teacher expectation for the workshop to provide
them with an understanding of how to use scientific inquiry to implement a
school-wide 6-8 scope and sequence was not substantially covered. The
expectations for teacher-generated end-products were adjusted to accommodate
the changes in schedule. For example, rather than completing a ready to use
standards-based unit plan, teachers were asked to complete and submit only the
outline for the plan. The re-planning,
rescheduling, and changing of specific content and requirements as a
consequence of this situation may have affected teachers’ perceptions of their
own accomplishments as well as the ability for the AWI Workshop to meet all of
its initial goals and objectives.