WEBVTT
1
00:00:03.300 --> 00:00:20.250
SASHA RAZOR: So, welcome to the session 7.2 the placement exams, and today we have three groups of presenters our first paper is presented by Professor is Isaac and Dr malia.
2
00:00:21.120 --> 00:00:25.440
SASHA RAZOR: Long back to us, and I will present the two.
3
00:00:26.160 --> 00:00:34.650
SASHA RAZOR: Speakers briefly so eve's as a as a professor in the department of languages and applied linguistics at the University of California Santa Cruz.
4
00:00:34.920 --> 00:00:43.380
SASHA RAZOR: Her research interests include second language acquisition heritage language development and language pedagogy in content based courses she.
5
00:00:43.830 --> 00:01:00.900
SASHA RAZOR: serves as an Executive Director of the language learning and Dr malia lumber to a sky is a professor of Spanish linguistics at cal poly pomona she conducts research and literacy development in Spanish heritage language learners welcome.
6
00:01:02.790 --> 00:01:03.210
Amàlia Llombart: Thank you.
7
00:01:03.570 --> 00:01:08.640
Eve Zyzik: Thank you, thank you very much so, shall I start and share my screen.
8
00:01:17.400 --> 00:01:19.080
let's just see.
9
00:01:20.340 --> 00:01:23.010
Eve Zyzik: Can everybody see my screen yes.
10
00:01:23.250 --> 00:01:32.400
Eve Zyzik: yeah all right wonderful, so thank you so much for being here on a Friday afternoon at least Friday afternoon here Pacific time.
11
00:01:34.080 --> 00:01:44.880
Eve Zyzik: i'm audio and I are really excited to tell you about this project that we've been working on for the last two years it's actually been a pandemic project, and I think it's kept us both sane.
12
00:01:46.230 --> 00:01:49.470
Eve Zyzik: With all of the events of the last two years, so.
13
00:01:51.180 --> 00:01:55.200
Eve Zyzik: This is what we hope to accomplish in our brief time today with you this afternoon.
14
00:01:55.830 --> 00:02:14.640
Eve Zyzik: I will talk about the design and the validation of the tests that we think could be useful for placement purposes, and also for research purposes and i'll be presenting the first part of that and then my colleague Amalia will present the data and the preliminary data that we have gathered.
15
00:02:17.880 --> 00:02:23.790
Eve Zyzik: So, just a brief background about vocabulary testing there's different formats of vocabulary tests.
16
00:02:24.330 --> 00:02:39.600
Eve Zyzik: Roughly divided into these four main types meaning recall form recall meaning recognition and form recognition the test that I will be presenting today is a meaning recognition test so we're in the green box there.
17
00:02:41.730 --> 00:02:54.990
Eve Zyzik: And the meaning recognition format looks something like this, so the learners will see a target word in this case it's in all caps they'll see the target word and there their goal is just to recognize.
18
00:02:55.770 --> 00:03:08.190
Eve Zyzik: The meaning of that word giving given several options right, so it could be a definition in the same language as you see here, or it could be an r1 equivalent from a list of choices.
19
00:03:11.370 --> 00:03:24.000
Eve Zyzik: If you're familiar with vocabulary testing, then surely you've seen something like this before we talk about high frequency vocabulary mid frequency and low frequency words, so this is.
20
00:03:25.110 --> 00:03:37.770
Eve Zyzik: From Schmidt and Schmidt 2014 but Paul nation has been writing about this for for many years, so in any language, we have a very small number of words that are extremely frequent.
21
00:03:38.130 --> 00:03:45.510
Eve Zyzik: So from the learners point of view they're very useful right you want to learn these words right away, and as quickly as possible.
22
00:03:46.170 --> 00:03:55.230
Eve Zyzik: When we're talking about heritage speakers, the situation changes, a little bit because heritage speakers are actually very likely to know.
23
00:03:55.830 --> 00:04:05.370
Eve Zyzik: Most of those high frequency words right so we're likely to get a ceiling effect if we only focus on those high frequency words so really to.
24
00:04:05.790 --> 00:04:23.220
Eve Zyzik: have some interesting information about their vocabulary knowledge, we really need to target words more at the MID frequency level okay So these are words roughly between 3009 thousand in in a rank order list of frequency.
25
00:04:26.580 --> 00:04:47.820
Eve Zyzik: So the test that we're that we have designed and validated tests five frequency levels from the 3000 level, all the way to eight from from worth 3001 all the way to the 8000 levels So these are listed here as 4k five K and so on.
26
00:04:51.030 --> 00:04:57.090
Eve Zyzik: The design of the test was based largely on the format of several tests that were developed.
27
00:04:58.260 --> 00:05:15.810
Eve Zyzik: By a team of researchers in the Netherlands, they were developed for French and English and we thought these tests were particularly promising they were based on recent frequency lists, they were tailored to the learners first language which, in their context was Dutch.
28
00:05:17.160 --> 00:05:22.530
Eve Zyzik: They tried to minimize guessing by adding an I don't know option to the format of the test.
29
00:05:22.950 --> 00:05:27.570
Eve Zyzik: And they aimed for a good representation of words within each frequency band so.
30
00:05:27.930 --> 00:05:37.920
Eve Zyzik: If you think back about the frequency bands, they have 1000 words and each band well how many do we sample I mean if we only sample five words from each frequency band.
31
00:05:38.430 --> 00:05:47.550
Eve Zyzik: that's not a very good representation right, so they aim for 30 words in each frequency level and that's That was our goal as well.
32
00:05:50.970 --> 00:06:04.440
Eve Zyzik: So let me tell you a little bit about where these words come from and in our test they come from a corpus it's the corpus that espanyol which you may be familiar with this is mark Davies corpus at.
33
00:06:05.340 --> 00:06:14.910
Eve Zyzik: brigham young university there's now a new sub feature of this corpus called word word and phrase which is particularly useful because.
34
00:06:15.300 --> 00:06:36.150
Eve Zyzik: word and phrase has a searchable frequency list of the top 40,000 words in Spanish so that's way more than we need right, but they are, they are rank ordered, as you can see here in this in this screenshot right i've taken a screenshot of words at the 5000 from 5273 and on so.
35
00:06:37.470 --> 00:06:45.660
Eve Zyzik: Each word is rank ordered in this way it's really important to mention that these are lemme based lists So what does that mean for us.
36
00:06:45.990 --> 00:07:08.010
Eve Zyzik: Well, it means that when we have a word like a lotto and lotto can be ice cream, or it can be frozen a lot over here as a noun is going to be listed separately from a level as frozen right, so, in fact, a lot of as a noun is is slightly more frequent than than adjective elavil.
37
00:07:09.240 --> 00:07:22.230
Eve Zyzik: Alright, so How did we designed this test How did we pick the words on the test, well, we started out with many more words than we knew we were going to need.
38
00:07:22.590 --> 00:07:30.120
Eve Zyzik: And the reason for that is because we knew we were going to be eliminating cognacs oh something happened can you still see my screen.
39
00:07:32.220 --> 00:07:33.150
Eve Zyzik: Okay sorry.
40
00:07:34.470 --> 00:07:52.680
Eve Zyzik: We knew we were going to be eliminating coordinates and so we started out with many more words, then, then we would need after eliminating cognitive we ended up with 40 words per per frequency band and then after pilot testing we narrowed it down to a smaller number of words per.
41
00:07:52.680 --> 00:07:53.220
frequency.
42
00:07:57.270 --> 00:07:57.780
Eve Zyzik: Another.
43
00:07:58.830 --> 00:08:10.290
Eve Zyzik: sort of calculation that goes into designing a vocabulary test is to determine not only which words we're going to include but which parts of speech and how these are going to be.
44
00:08:11.280 --> 00:08:23.130
Eve Zyzik: distributed right so in Spanish, these are not equal categories there's many more nouns in Spanish than there are adjectives right roughly twice.
45
00:08:23.490 --> 00:08:36.930
Eve Zyzik: The amount of nouns adjectives in any given frequency band and fewer verbs so we tried to have our test reflect this this reality of how parts of speech are distributed in the language.
46
00:08:39.300 --> 00:08:50.280
Eve Zyzik: So I already mentioned this, but i'll go into a little bit more detail here we we knew we were going to eliminate Spanish English cod mates How did we do that, well, we did it individually at first.
47
00:08:51.060 --> 00:09:03.930
Eve Zyzik: Just by ourselves, we went through all of the words we have chosen and any we flag any word that we thought was a coordinate and then we discuss these together and resolved any disagreement, so this was.
48
00:09:04.980 --> 00:09:11.580
Eve Zyzik: Also, a lot of fun to do we were surprised by how many carbonates there were and how many words we had to eliminate.
49
00:09:12.390 --> 00:09:21.060
Eve Zyzik: We also eliminated problematic items that were problematic and in a different way right, so we really didn't want to have any words that were.
50
00:09:21.720 --> 00:09:31.650
Eve Zyzik: Particular to one dialect one variety of Spanish we eliminated words that are polygamist, which was also really quite a shame, but.
51
00:09:32.160 --> 00:09:42.600
Eve Zyzik: alyssa miss words had to go, because we needed to have a clear definition for each of our words so post them us words in that sense would be a problem.
52
00:09:43.260 --> 00:09:57.960
Eve Zyzik: We also eliminated words that had a high frequency route and so with this example with makoto menthol, this is a word that we would have liked to have used, for example, but the route mccord is very high frequency right so.
53
00:09:58.920 --> 00:10:12.600
Eve Zyzik: That would have influenced our art calculation of frequency there and then we also eliminated words that were highly specialized being either medical terminology, legal or or religious terminology.
54
00:10:15.690 --> 00:10:33.870
Eve Zyzik: Alright, so once we have selected our words we began writing definitions, this was also, I think a lot of fun for us and we enjoy collaborating on this part, so we just began with the dictionary definitions, but those dictionary definitions needed to be reduced, they needed to be simplified.
55
00:10:35.130 --> 00:10:42.390
Eve Zyzik: Sometimes we opted for including a synonym in the definition, rather than along dictionary definition so.
56
00:10:42.870 --> 00:11:01.620
Eve Zyzik: Another part of writing the definitions was that, since this is a modeling will test right the definitions are also in Spanish, we needed to make sure that all the words used in the definitions were also high frequency words or coordinates that we assume that the speakers know.
57
00:11:03.840 --> 00:11:20.190
Eve Zyzik: So the final product is 150 items test that has this format that you see here, so the the target word is written in all caps there's no metal language here right so it's the nod.
58
00:11:21.090 --> 00:11:30.630
Eve Zyzik: Is a verb but we're not tagging it as a verb we're just putting it out there, and there are there's only one correct answer in this case it's be.
59
00:11:31.110 --> 00:11:41.580
Eve Zyzik: The others are the other answers are distractors and then there's an I don't know option at the bottom, remember, we did this in order to reduce guessing and.
60
00:11:42.570 --> 00:11:57.360
Eve Zyzik: Maybe one final note, really quickly is that we decided to leave all the map adjectives and masculine form rather than complicating it visually with like an old slash a ending for those adjectives that that helped that.
61
00:11:59.610 --> 00:12:11.310
Eve Zyzik: So i'm going to turn it over to i'm going to stop sharing my screen i'm going to turn it over to my colleague Amalia who will go through the research questions and and the data.
62
00:12:24.210 --> 00:12:36.690
Amàlia Llombart: Okay, so i'm a research questions were first industrial have a test have a scalable profile, so, in other words do Okay, the speaker score highest on the 4000 frequency band.
63
00:12:36.870 --> 00:12:42.210
Amàlia Llombart: Then lower on the 5000 frequency by then lower still in the 6000 and so on.
64
00:12:42.900 --> 00:12:51.810
Amàlia Llombart: And the second question is best performance on the covetous correspond to different levels in the Spanish language dominance.
65
00:12:53.640 --> 00:13:01.380
Amàlia Llombart: So we've collected or data through two instruments for despair bilingual language profile, I mean in this order the bilingual language profile.
66
00:13:01.830 --> 00:13:12.720
Amàlia Llombart: Which is a questionnaire design by birdsong garcon and meanwhile that has four components questions in divided into four components language, history language to use.
67
00:13:13.440 --> 00:13:37.860
Amàlia Llombart: Language proficiency saturated and language attitudes and it gives you a value that ranges between negative to 18 and positive to 18 so we're value of zero or around serum will be balanced, spelling law and positive numbers are English dominance and then negative numbers Spanish dominance.
68
00:13:39.540 --> 00:13:56.130
Amàlia Llombart: And then we give the cover test right and we've had 89 participants, which were most of them were enrolled in as far as for Spanish speakers courses and there were a few students enrolled in.
69
00:13:57.960 --> 00:14:05.310
Amàlia Llombart: In other Spanish courses, I mean a Spanish program courses that didn't define themselves as native okay that speakers.
70
00:14:07.110 --> 00:14:21.720
Amàlia Llombart: So first, then we looked at the reliability of the items in each in each frequency band, and the values were two very good and excellent the lowest one was in the 4000 like.
71
00:14:23.220 --> 00:14:31.110
Amàlia Llombart: group, it was a point at nine the highest in the 5000 point 94 so I mean.
72
00:14:32.970 --> 00:14:44.550
Amàlia Llombart: Then I here we can see the descriptive statistics, with the results that the participants had on the on the test, remember that we have 30 items, so this is not percentage.
73
00:14:45.570 --> 00:15:17.190
Amàlia Llombart: Based on 30 items image frequency band so for 4000 we had an average of 2121 58 then in the 5014 14.2 out of 36,000 you see 15 or 37,010 out of 30 and 1000 just they may be higher 18 or a fairy and the total which is 150 items, the average score was 70 7050.
74
00:15:19.230 --> 00:15:30.630
Amàlia Llombart: So to see there were differences we repeated measures are Nova with a greenhouse is a correction and it shows that the mean vocabulary score different significantly between or frequency levels.
75
00:15:31.920 --> 00:15:49.350
Amàlia Llombart: So, then, we did a post hoc otherwise test with minority corrections and all differences were significant, except for the difference between five 5006 thousand and between 7000 and a person, and we can see more clearly here on.
76
00:15:52.230 --> 00:15:54.390
Amàlia Llombart: Here, as we can see that Sir make.
77
00:15:55.650 --> 00:16:10.950
Amàlia Llombart: A strong drop a big drop between 4005 thousand and between six and seven, but here those two are the ones with the red lines indicate the ones that were not significant differences.
78
00:16:11.760 --> 00:16:20.580
Amàlia Llombart: So what we did, then was the second computation and with the same as results within seven to them, competition and now we completed two new variables.
79
00:16:21.120 --> 00:16:35.070
Amàlia Llombart: We combine the results for five 5000 6000 and then from 7000 8000, of course, we divided by a beta so we have a filter the maximum score of.
80
00:16:35.880 --> 00:16:52.080
Amàlia Llombart: 38 cents, I mean 30 and here we have three variables, now that 4000, which of course has the same score right to anyone, and then the score for the combine the combination of five and six was 13 and the combination between.
81
00:16:53.070 --> 00:17:14.670
Amàlia Llombart: off seven and 1000 frequency band was then, and here, so we did again and repeat and measured so Nova and of course it was significantly different, and their boss hopper wasters tests, I showed that now all contracts were significant and we can see here well clearly.
82
00:17:16.290 --> 00:17:20.340
Amàlia Llombart: that these three variables different and is.
83
00:17:21.510 --> 00:17:26.190
Amàlia Llombart: Lower score lowest scores, as the frequency levels are higher.
84
00:17:29.160 --> 00:17:35.160
Amàlia Llombart: Higher frequency know higher well and therefore thousands right and here.
85
00:17:36.450 --> 00:17:57.660
Amàlia Llombart: We are so look at the bill discourse, with the only with 17 participants and the range was between negativity nine and one positive 140 remember that negative numbers show more Spanish dominance and the positive numbers more English dominance and the mean was.
86
00:17:58.740 --> 00:18:02.970
Amàlia Llombart: 43 positives are still in the English more English dominant.
87
00:18:03.990 --> 00:18:07.380
Amàlia Llombart: And then I will be there be lpn cava.
88
00:18:08.400 --> 00:18:16.890
Amàlia Llombart: correlation and we obtain a strong negative correlation of negative point 714.
89
00:18:18.360 --> 00:18:29.070
Amàlia Llombart: So, going back to your research questions, the first one was does the characters have a scalable profile like 4000 to 5000 6000 and, yes, and not.
90
00:18:29.550 --> 00:18:43.620
Amàlia Llombart: Because, if you remember the contrast between 5000 6000 and then between seven and eight was not significant so this suggest that we can collapse it test into three levels, four, five and six and seven and eight.
91
00:18:45.360 --> 00:18:51.510
Amàlia Llombart: The second question was does performance on the cover test correspond to different levels of Spanish language permanence.
92
00:18:52.020 --> 00:19:04.170
Amàlia Llombart: And yes, definitely there is a strong negative correlation there higher the plp score, the lower the score on data so as finest prominence correlates with vocabulary knowledge.
93
00:19:04.860 --> 00:19:19.020
Amàlia Llombart: And this is consistent with previous findings that will be built this dominant score has been found to correlate with our vocabulary efficiency measures, for example, and knowledge of qualifications, a seismic so in.
94
00:19:21.000 --> 00:19:21.690
Amàlia Llombart: 2021.
95
00:19:24.090 --> 00:19:32.160
Amàlia Llombart: So our future directions First, we want to revise it best to reduce the items to 30 in it's bad, so we will have this city.
96
00:19:32.700 --> 00:19:48.600
Amàlia Llombart: Vance 4000 with 38 items then 30 items for the five and six and then 38 and 27 and eight, which will give us a total of 90 items, instead of 150 which I think we've seen, we would be more manageable.
97
00:19:49.980 --> 00:20:04.230
Amàlia Llombart: Then Second, we want to expand the population of participants to first generation immigrants, so the parents of our heritage language learners we would modify the test to include audio so you will have both everything an rv at the same time.
98
00:20:05.880 --> 00:20:19.560
Amàlia Llombart: Then we also want to be able to create groupings of haters language learners based on test scores so so see that maybe this could be used for placement purposes, in view of Delhi.
99
00:20:20.970 --> 00:20:30.270
Amàlia Llombart: And finally, once explore correlations between vocabulary knowledge based on camera and other measures of literacy, such as reading comprehension.
100
00:20:31.650 --> 00:20:40.470
Amàlia Llombart: So this is all and we're grateful, thank you to all of you for being here and listening and we are so grateful to everyone who help us recruit participants.
101
00:20:41.040 --> 00:20:52.920
Amàlia Llombart: As well as their colleagues who provided feedback on preliminary versions of the test and, if you will, like to use this test in your response rates and which underscores please contact us thank you.
102
00:20:54.630 --> 00:20:56.610
Amàlia Llombart: When to stop sharing maybe.
103
00:20:57.120 --> 00:21:03.120
SASHA RAZOR: yeah stop sharing, thank you both Amalia and he for a wonderful wonderful.
104
00:21:03.120 --> 00:21:07.380
SASHA RAZOR: presentation at our would like to remind all our participants that.
105
00:21:07.680 --> 00:21:18.000
SASHA RAZOR: As you're watching this presentation and there is a question that you have please write it down and posted in the chat on the much more easier to shape our discussion around these questions.
106
00:21:18.360 --> 00:21:26.580
SASHA RAZOR: And the second group of presenters that we have we have three presenters joining us from new Mexico and you see them on the screen, and let me.
107
00:21:27.180 --> 00:21:42.060
SASHA RAZOR: quickly say who they are, and their biographies so Patricia McGregor Mendoza hi Patricia is a professor of Spanish and linguistics that has been teaching at new Mexico State University since 1995.
108
00:21:42.510 --> 00:21:54.570
SASHA RAZOR: She has published on language and language attitudes in bilingual communities language policy and educational settings language and immigration policies language assessment and the teaching of.
109
00:21:55.020 --> 00:22:08.790
SASHA RAZOR: Languages and linguistics online welcome Patrice yeah and the next presenter is Gabriella Maria know who is an associate professor of Spanish in the department of languages and linguistics at new Mexico State University him.
110
00:22:09.270 --> 00:22:22.830
SASHA RAZOR: As a native of the borderlands and missy missy a valley area upbringing, has been very influential to her teaching and research interests her scholarly expertise is in the area of pedagogy and.
111
00:22:23.160 --> 00:22:31.530
SASHA RAZOR: methodology for heritage language speakers, which includes language, the socialization languages and quantify and acquisitions of Spanish.
112
00:22:31.770 --> 00:22:44.190
SASHA RAZOR: In the US her research interests expand to the area of water studies and chicano Mexican border literature welcome Gabriella and the third presented today is Jeff lauren well.
113
00:22:44.880 --> 00:22:55.740
SASHA RAZOR: Who would like to be called Prof Jaffa our new brackets here, who has been at new Mexico State University, for more than 20 years Professor Jeff from.
114
00:22:56.160 --> 00:23:04.920
SASHA RAZOR: Academic interest include of finding ways to continue to engage students and being an active part of their own learning process.
115
00:23:05.370 --> 00:23:13.950
SASHA RAZOR: applications of technology and language learning online education and helping students have a successful experience in their learning environments.
116
00:23:14.160 --> 00:23:22.950
SASHA RAZOR: wherever that may be in his free time he enjoys spending time with his family doing what they were the kids would love to do so welcome all and let's begin your presentation.
117
00:23:25.980 --> 00:23:26.700
SASHA RAZOR: let's see.
118
00:23:27.990 --> 00:23:28.470
got it.
119
00:23:31.110 --> 00:23:34.350
Jeffrey Longwell: Okay we're almost there we're almost there.
120
00:23:36.060 --> 00:23:36.750
From the.
121
00:23:44.520 --> 00:23:44.790
Okay.
122
00:23:46.740 --> 00:23:52.200
Jeffrey Longwell: Okay, well, thank you, we are very excited to be part of this panel.
123
00:23:53.460 --> 00:24:08.100
Jeffrey Longwell: I was just thinking of so many things that we're aligning with with what we're doing with with what the previous presenters presented so today we're going to be talking about our continuing efforts also, as we have continued on through the pandemic.
124
00:24:09.390 --> 00:24:18.990
Jeffrey Longwell: Looking at adding items to our already well functioning placement exam and our topic is going to be examining as a chill us in his performance.
125
00:24:19.440 --> 00:24:36.180
Jeffrey Longwell: On linguistic items for placement implications and i'm pretty sure McGregor Mendoza and I teach courses in Spanish and in linguistics but if it says span in front of it, I teach it in Spanish is if it says laying in front of it, I teach English.
126
00:24:37.620 --> 00:24:47.010
Jeffrey Longwell: guardian guardiola Moreno i'm the director of the heritage language learner program and I am also responsible for the pedagogy and methodology courses for heritage language learners.
127
00:24:47.700 --> 00:25:00.300
Jeffrey Longwell: And Jeff long well i'm the coordinator for the second language program excuse me and teach courses in second language pedagogy and also integration technology and in language learning.
128
00:25:01.980 --> 00:25:10.800
Jeffrey Longwell: So just to give you a little bit of a background, we have a traditional second language program here at new Mexico State University, with the typical Spanish 123 and four.
129
00:25:11.130 --> 00:25:20.970
Jeffrey Longwell: That are offered in many other institutions, we also have grammar and composition courses for non native speakers, we have to say that in all of our.
130
00:25:21.480 --> 00:25:31.110
Jeffrey Longwell: classes, we also have heritage learners that are interspersed in this is not because we're not doing an adequate job of placement and testing, because we do, but.
131
00:25:31.500 --> 00:25:42.000
Jeffrey Longwell: It does occur for other reasons, like poor advising from other departments or simple self perception from the heritage learners themselves.
132
00:25:44.130 --> 00:25:52.650
Jeffrey Longwell: So our program here is pioneering the nation serving a diverse population of learners embracing different heritage varieties of Spanish handed down through.
133
00:25:52.890 --> 00:26:04.590
Jeffrey Longwell: generations as well as learners who have spent more or some or all of their formative years in Mexico, for this reason our program embraces both heritage language learners and native speakers.
134
00:26:05.190 --> 00:26:10.530
Jeffrey Longwell: Many students come into the program with a high level of speaking ability, but like the middle linguistic knowledge.
135
00:26:10.980 --> 00:26:19.530
Jeffrey Longwell: We offer beginning courses levels at the 100 level and to intermediate courses at the 200 level, we also offer grammar and composition courses at the 300 level.
136
00:26:20.100 --> 00:26:29.070
Jeffrey Longwell: We developed a successful placement exam using sadness that subjunctive items and an innovative reading test, however, we also seek to develop new items.
137
00:26:29.370 --> 00:26:39.840
Jeffrey Longwell: to integrate into that test and hence have piloted a number of new items, based on the format and content of the already established ones we continue to monitor their performance as well.
138
00:26:41.610 --> 00:26:52.200
Jeffrey Longwell: We also use as our foundation, the work of hosting and basic theories about basic language, cognition and higher language, cognition vlc refers to the basic elements of Spanish.
139
00:26:52.410 --> 00:27:01.050
Jeffrey Longwell: And how its put together that heritage learners and native speakers will know intuitively intuitively due to their earlier exposure to the language.
140
00:27:01.770 --> 00:27:06.570
Jeffrey Longwell: Second language learners will only have access to higher language, cognition and will take time.
141
00:27:06.900 --> 00:27:16.050
Jeffrey Longwell: And effort to learn the things that heritage learners will know instinctively in addition to the Beatles enough knowledge heritage learners also have to.
142
00:27:16.350 --> 00:27:31.920
Jeffrey Longwell: Have the HST knowledge, however, that knowledge will vary, given the variety of experiences in and out of the classroom second language learners can potentially become proficient in in i'm sorry in H lc knowledge, over time, with much practice.
143
00:27:34.680 --> 00:27:42.570
Jeffrey Longwell: Some of the differences in which heritage learners process said in a star illustrate these differences research has shown that heritage.
144
00:27:42.990 --> 00:27:49.320
Jeffrey Longwell: learners developed early sensitivity to the semantic distinctions of the two verbs and carry that through adulthood.
145
00:27:50.100 --> 00:28:02.700
Jeffrey Longwell: On the other hand, second language learners did not begin to develop distinct crunch distinctions between the two verbs until late in their first year or second year of study, and then they progress slowly.
146
00:28:04.380 --> 00:28:11.490
Jeffrey Longwell: Similarly, the subjunctive is another purple area of study in terms of vlc and a Chelsea.
147
00:28:12.090 --> 00:28:27.420
Jeffrey Longwell: Research shows that heritage learners begin to produce subjunctive forms and proceed through various stages of acquisition, based on the complexity of the discourse, to which they're exposed that's the context guides the development of the use of the subjunctive.
148
00:28:28.470 --> 00:28:44.550
Jeffrey Longwell: The lower production of farms along generational lines of heritage learners has been unfortunately missed characterized by some as incomplete, rather than a signal that learners simply need more exposure to the context in which these forms might be needed.
149
00:28:45.660 --> 00:28:51.780
Jeffrey Longwell: On the other hand, the path of mastery for a second language learners as long and begin somewhat late in their learning.
150
00:28:52.320 --> 00:29:06.690
Jeffrey Longwell: The forms and their semantic fun functions are somewhat opaque to these learners they instead rely on specific syntactic triggers like irregular verb forms or verbs evolution to begin to produce the subjunctive.
151
00:29:09.600 --> 00:29:19.230
Jeffrey Longwell: to tap into the literacy skills we use forms that replicates the type of literacy activities in which heritage learners generally engage to express themselves in text form.
152
00:29:19.680 --> 00:29:38.160
Jeffrey Longwell: So non traditional texts such as social media exchanges are shorter recognizable meaningful and emphasized communication, we have found that texts in these forms represent a less intimidating and more accessible means of assessing literacy skills in all learners.
153
00:29:40.200 --> 00:29:48.120
Jeffrey Longwell: So, because of the contracting ways in which learners acquire and learn and process, the information on said, and this that and the various areas of the subjunctive.
154
00:29:48.450 --> 00:30:00.180
Jeffrey Longwell: These areas are fruitful, to help us determine how to place learners in our two programs Similarly, we found that the innovative reading tests are an effective way to examine existing literacy skills.
155
00:30:00.510 --> 00:30:09.000
Jeffrey Longwell: Collectively we found that the performance on these items are very helpful and helping us place learners and different backgrounds and different abilities in our Program.
156
00:30:09.480 --> 00:30:15.000
Jeffrey Longwell: We presented recently on the success of these types of items which we summarize briefly here.
157
00:30:15.360 --> 00:30:23.910
Jeffrey Longwell: Overall we've confirmed that, for our student population these items are useful and discerning differences between heritage language and second language populations.
158
00:30:24.210 --> 00:30:36.690
Jeffrey Longwell: Moreover, we found that read the reading items we've developed, which are based on the literacy practices in which heritage learners engage on a daily basis are more effective in tapping into the literacy skills that they have in Spanish.
159
00:30:36.990 --> 00:30:44.070
Jeffrey Longwell: than our traditional readings and that they still provide distinctions between second language populations and heritage language population.
160
00:30:46.710 --> 00:30:58.230
Jeffrey Longwell: So sorry so looking at the test outcomes can give us an idea on how participants performed on the test, much like looking at how people perform when they drive a car.
161
00:30:58.590 --> 00:31:11.880
Jeffrey Longwell: Here we are taking a different perspective by looking at item analyses item analyses examine how each item itself is performing and whether or not each item aligned with others to produce a reliable outcome.
162
00:31:12.240 --> 00:31:17.520
Jeffrey Longwell: So with item analyses we're checking under the hood for our assessment to examine its inner workings.
163
00:31:17.730 --> 00:31:33.900
Jeffrey Longwell: We want to see whether the items are too easy or too difficult, whether they can distinguish between learners at different levels and whether or not they're reliable consistent measures and how the options that we offer for learners to choose to choose from our performing overall.
164
00:31:36.000 --> 00:31:52.230
Jeffrey Longwell: So that leads us to our overall our overarching research question, which is how well are the items themselves performing with respect to their potential to aid us in placing learners some of the underlying sub questions include are they have appropriate difficulty.
165
00:31:53.370 --> 00:32:00.690
Jeffrey Longwell: Do they all do they allow us to distinguish between learners of different ability levels and are they reliable.
166
00:32:01.830 --> 00:32:10.290
Jeffrey Longwell: Additionally, we looking at the distractors are the distractors being selected as students, see the items in the placement exam.
167
00:32:12.630 --> 00:32:18.120
Jeffrey Longwell: Just a quick overview of our our methods in them all up on the screen here.
168
00:32:21.720 --> 00:32:29.250
Jeffrey Longwell: So far, we've collected data over three semesters in all of the 100 to 300 level courses in both of our programs.
169
00:32:29.760 --> 00:32:36.030
Jeffrey Longwell: You can see that among those enrolled in our second language courses, we had a higher participation, the classes at the lower end.
170
00:32:36.720 --> 00:32:43.620
Jeffrey Longwell: The opposite is the case for the heritage language courses where participants were more frequently drawn from the 300 level courses.
171
00:32:44.310 --> 00:32:55.260
Jeffrey Longwell: In addition to content items we also included questions pertaining the individuals backgrounds, particularly with reference to whether or not they were exposed to Spanish in their home.
172
00:32:56.430 --> 00:33:08.310
Jeffrey Longwell: We do this because many individuals who are heritage learners are not always confident about their Spanish abilities and we will often and will often not identify as Spanish speakers if asked directly.
173
00:33:08.910 --> 00:33:17.730
Jeffrey Longwell: We find that, for a variety of reasons, students, that should be in our heritage language classes often enroll in our second language courses.
174
00:33:20.970 --> 00:33:30.840
Jeffrey Longwell: The three big tests we include are known as item difficulty which evaluates how easy or difficult it is for participants to select the right answer.
175
00:33:31.410 --> 00:33:42.540
Jeffrey Longwell: The discrimination index examines the items ability to distinguish between learners by comparing the performance of top scores on the item with that of the low scores on the item.
176
00:33:43.440 --> 00:33:53.160
Jeffrey Longwell: Reliability measure in this case the point by serial shows us how consistently match the items performances with that of the overall outcome of the test.
177
00:33:53.970 --> 00:34:05.130
Jeffrey Longwell: will complement our examination of these measures with distract or analyses to check the performance of each of the distractors or wrong options that we include on each item.
178
00:34:07.350 --> 00:34:18.300
Jeffrey Longwell: So now, the fun part the numbers so item difficulty is the proportion of people that select the correct answer for placement, we want the items that represent.
179
00:34:18.690 --> 00:34:29.460
Jeffrey Longwell: A range of difficulty levels, but if they're too easy or too hard they can't really help us determine abilities, so we want them to generally fall somewhere between point two, and point eight five.
180
00:34:30.300 --> 00:34:39.600
Jeffrey Longwell: Discrimination index looks at the proportion of correct answers from among the top 27% of scores and and subtracts from that.
181
00:34:40.050 --> 00:34:47.460
Jeffrey Longwell: The proportion of correct answers from the bottom 27% of scores, to see if the items can help distinguish learners have different abilities.
182
00:34:47.880 --> 00:34:55.890
Jeffrey Longwell: We can't really count on anything at the point to level or below moderate correlation start around point to one.
183
00:34:56.310 --> 00:35:03.390
Jeffrey Longwell: And anything that point four oh and above represents, you know, a strong ability to discriminate between murders.
184
00:35:03.780 --> 00:35:12.210
Jeffrey Longwell: The point by serial calculate the correlation between the score on the individual item and the total score to check for consistency.
185
00:35:12.540 --> 00:35:18.240
Jeffrey Longwell: The parameters that guide the evaluation of items on this measure are similar to those of the discrimination index.
186
00:35:18.750 --> 00:35:26.940
Jeffrey Longwell: But it's important to note that these statistics are used by test designers to signal that items are either working well or to signal areas.
187
00:35:27.270 --> 00:35:35.370
Jeffrey Longwell: that need to be looked at more closely for flaws, no one statistic is examined in isolation they're used in conjunction with one another.
188
00:35:35.730 --> 00:35:50.250
Jeffrey Longwell: So, and this type of analysis is a necessary step to take when we're looking we're kicking the tires and checking under the hood of our our test so we're going to look at the numbers of all of these analyses in turn.
189
00:35:51.660 --> 00:36:00.930
Jeffrey Longwell: So regarding item difficulty for sending to start, we see that most of the items are performing within the expected parameters signaled by those blue horizontal lines.
190
00:36:01.320 --> 00:36:09.270
Jeffrey Longwell: As predicted by the notion of the basic language, cognition we see that for for heritage learners these items are fairly basic.
191
00:36:09.660 --> 00:36:18.570
Jeffrey Longwell: But for the most part there's a range of difficulty, moreover, we can see that for different student types the item difficulty varies.
192
00:36:18.930 --> 00:36:34.440
Jeffrey Longwell: Offering confirmation that said in a stock items can help us in determining placement, the items that peak above the top line, let us know that these items might be too easy for learners so we'll keep our eye on them as we explore further.
193
00:36:35.550 --> 00:36:43.770
Jeffrey Longwell: Our subjunctive item difficulty our outcomes tell a similar tale, we can observe a nice range of difficulty levels and.
194
00:36:44.340 --> 00:36:55.200
Jeffrey Longwell: and note that the items challenge each type of learners differentially We note also that some of the items show that they're too difficult for the lt learners, however.
195
00:36:55.590 --> 00:37:10.680
Jeffrey Longwell: Given that the majority of the lt learners in our pool were drawn from the first two semesters they may not have had that much exposure to the subjunctive still this outcome tells us that we need to keep our eye on these items, with regard to our learners.
196
00:37:11.970 --> 00:37:17.010
Jeffrey Longwell: Looking at our item difficulty for the reading.
197
00:37:18.270 --> 00:37:31.620
Jeffrey Longwell: also tells you know, a similar tale and gives us strong confirmation that these items are working, we need to keep in mind that these were these readings were designed with our heritage language learners in mind.
198
00:37:32.130 --> 00:37:46.920
Jeffrey Longwell: But we see that they're not only that they're have appropriate levels of difficulty, not only for these heritage learners but also revealed that they can be useful for the placement of lt learners as well we do note that one reading.
199
00:37:48.420 --> 00:37:56.910
Jeffrey Longwell: appears to be a little bit more challenging for lt learners suggesting again that we have to look a little bit more closely at how we're wording, some of those items, perhaps.
200
00:37:58.710 --> 00:38:07.200
Jeffrey Longwell: So, looking at the discrimination index recall that this allows us to tell how effective the items are in determining differences of ability.
201
00:38:07.650 --> 00:38:16.560
Jeffrey Longwell: or minimum marked by that look that red line is point two, and anything above point for marked by the green line is strong.
202
00:38:17.010 --> 00:38:26.910
Jeffrey Longwell: So here we see that most of our items meet these parameters, indicating that yes, we can use these to distinguish between levels of ability for learners.
203
00:38:27.420 --> 00:38:33.510
Jeffrey Longwell: One said item stands out for its lack of discrimination, ability and that item.
204
00:38:33.870 --> 00:38:41.400
Jeffrey Longwell: was the one that we saw earlier might be a little bit too easy for all groups, which means that it has little hope of distinguishing between.
205
00:38:41.730 --> 00:38:53.250
Jeffrey Longwell: learner abilities of any of the learner types, so our options as chest designers are either to rewrite the item and retest it or simply excluded from further consideration.
206
00:38:55.230 --> 00:39:12.090
Jeffrey Longwell: The discrimination index for our subjunctive items also confirms that our items are for the most part, able to distinguish between learner types, we see that a couple of items are too difficult for our learners which may again be reflective of characteristics of our learner pool.
207
00:39:13.110 --> 00:39:23.220
Jeffrey Longwell: The discrimination index for our reading items is also strong indicating that the items work well at being able to distinguish between learners at different at a variety of skill levels.
208
00:39:23.610 --> 00:39:32.820
Jeffrey Longwell: And also we'll just remember that these are items that were designed with our heritage learners in mind, but they're also useful for our second language learners.
209
00:39:34.200 --> 00:39:40.740
Jeffrey Longwell: The reliability calculations confirm that most of the set in a star items are also highly reliable.
210
00:39:41.340 --> 00:39:54.840
Jeffrey Longwell: This outcome, means that the items are all aligned with one another on testing in the same area only one item isn't aligned with the others which, given the results of the previous analyses isn't surprising at this point.
211
00:39:56.850 --> 00:40:06.210
Jeffrey Longwell: Looking at our reliability of the subjunctive items which also confirmed by and large, meaning that they're consistently measuring the same construct.
212
00:40:06.630 --> 00:40:17.730
Jeffrey Longwell: Only again two items fall slightly below our minimum minimum threshold for the lt learners and again we think this is this could be resolved by adding to our subject pool.
213
00:40:19.200 --> 00:40:30.810
Jeffrey Longwell: So the reliability for all three readings, is also very strong this outcome again confirms that the reading items can be relied upon to measure reading abilities for different types of learners.
214
00:40:32.370 --> 00:40:41.850
Jeffrey Longwell: Sorry i'm I know i'm moving fast there's a lot of stuff on here, but we wanted to just share all of our lovely things, these are the are distract or analyses.
215
00:40:42.300 --> 00:40:49.020
Jeffrey Longwell: And these give us an overall sense of how well the non correct options that we provided learners are working, that is.
216
00:40:49.260 --> 00:40:59.490
Jeffrey Longwell: it's a measure of whether or not the distractors are appealing or sensed as plausible for for different levels of learners there's no hard and fast science.
217
00:40:59.790 --> 00:41:11.070
Jeffrey Longwell: To this, if you notice that some of the distractors are not being selected very often or not at all, it means that those options are likely discounted from the learners consideration.
218
00:41:11.490 --> 00:41:21.990
Jeffrey Longwell: That can affect item difficulty, as it means that the path to the correct answer is made easier through simple elimination and not through skill or ability.
219
00:41:22.590 --> 00:41:37.410
Jeffrey Longwell: So here in these graphs graphs for the sake of convenience of comparison we've set all the correct answers a which is characterized by the dark blue portions at the bottom of each bar.
220
00:41:37.980 --> 00:41:58.350
Jeffrey Longwell: The variety of colors on the bars here indicate how often the other options are being selected, so we can see that heritage learners are more proficient in finding the correct answers, overall, we do notice the lack of selection of alternatives for one question.
221
00:41:59.610 --> 00:42:09.540
Jeffrey Longwell: lets us know that learners can easily dismiss the distractors which, in conjunction with the results of the other analyses signals that that item is problematic for our purposes.
222
00:42:10.920 --> 00:42:15.450
Jeffrey Longwell: Here again, we see that the distractors for the reading items are productive.
223
00:42:15.780 --> 00:42:28.050
Jeffrey Longwell: Again, suggesting a sound design and we can see that the different learner groups are patterning differently in their selection of the distractors which also suggest that the selections are being based on ability.
224
00:42:28.500 --> 00:42:34.590
Jeffrey Longwell: With the lt learners we see that they are frequently selecting the no say option so they're tapping out.
225
00:42:35.070 --> 00:42:51.090
Jeffrey Longwell: And that's indicated by the light blue proportions at the top of each bar that indicates that many of them find the readings more challenging than do the heritage language learners but again it's confirmed by the other results they're still useful in helping us determine things.
226
00:42:57.690 --> 00:43:05.700
Jeffrey Longwell: Okay, so here's some things that we'd like you to take away from this presentation.
227
00:43:06.390 --> 00:43:20.760
Jeffrey Longwell: Number one the item analysis confirm our findings said in a star the subjunctive and non traditional readings are appropriate as placement items, since they are areas that reveal differing levels of abilities in different types of learners.
228
00:43:21.990 --> 00:43:27.720
Jeffrey Longwell: The items themselves are structured appropriately, with the exception of the one item which we spend time on.
229
00:43:29.160 --> 00:43:40.980
Jeffrey Longwell: Finally, heritage learners and second language learners responses pattern differently lending support to philistines enzyme six notions of vlc and a Chelsea.
230
00:43:42.720 --> 00:43:44.040
Jeffrey Longwell: So where do we go from here.
231
00:43:54.630 --> 00:43:55.260
Jeffrey Longwell: So.
232
00:43:56.460 --> 00:44:07.590
Jeffrey Longwell: are very high results on the discrimination and reliability may result from characteristics of our participant pool meaning, you know, we had a lot of.
233
00:44:08.310 --> 00:44:12.060
Jeffrey Longwell: heritage learners at the 300 level and second language learners from the.
234
00:44:12.690 --> 00:44:23.100
Jeffrey Longwell: 100 level, we need to fill in more learners at the two and 300 levels from the second language program and more 102 hundred level learners from the heritage Program.
235
00:44:24.000 --> 00:44:37.230
Jeffrey Longwell: We anticipate that adding more learners from these groups will reduce the strength of the results of the item difficulty discrimination index and reliability, but not fundamentally alter the patterning of their results.
236
00:44:39.120 --> 00:44:46.620
Jeffrey Longwell: we're also interested in finding out more about the experiences of our heritage learners that filter into our second language courses.
237
00:44:47.250 --> 00:44:57.660
Jeffrey Longwell: Were curious about why they enter what motivates they're staying and whether their experiences in a second language learning environment alters their outcomes.
238
00:45:00.720 --> 00:45:09.870
Jeffrey Longwell: So what you should know you can develop a placement exam that is relevant to your program and that your learners and your learners and that is based on research.
239
00:45:10.080 --> 00:45:15.060
Jeffrey Longwell: We found that a few simple items can be designed to detect differences between learners of different types.
240
00:45:15.240 --> 00:45:22.890
Jeffrey Longwell: it's not necessary to develop a long test that assesses a lengthy inventory of knowledge in order to find what courses are best suited for our learners.
241
00:45:23.250 --> 00:45:29.580
Jeffrey Longwell: As you develop items we encourage you to do so, keeping in mind the expertise of the.
242
00:45:30.000 --> 00:45:37.770
Jeffrey Longwell: Of the Spanish heritage learner especially their natural acquisition of language in mind at first as well, rather than consider grammatical norms or.
243
00:45:38.250 --> 00:45:46.860
Jeffrey Longwell: Expectations also you don't need to purchase at access to stats programs much of the item analysis can be accomplished with some minimal skills in excel.
244
00:45:47.250 --> 00:45:57.150
Jeffrey Longwell: testing and assessment of the placement exam are some of the critical ways that we ensure that our program our programs are not only meeting their goals but also serving our learners as well.
245
00:45:59.640 --> 00:46:02.700
Jeffrey Longwell: Finally, we'd like to to express that we have.
246
00:46:03.780 --> 00:46:04.140
thrown.
247
00:46:07.290 --> 00:46:16.740
Jeffrey Longwell: we'll just do a little commercial commercial for our program and we have an ma in Spanish program offered face to face and online here at new Mexico State University, which offers.
248
00:46:17.130 --> 00:46:28.050
Jeffrey Longwell: Teaching assistantships for students in both heritage language programs and second language program So if you are your students are interested, please have them reach out to us.
249
00:46:30.810 --> 00:46:33.630
Jeffrey Longwell: And hit us yes Thank you so much.
250
00:46:34.560 --> 00:46:50.460
SASHA RAZOR: Thank you, thank you for a wonderful presentation Patricia gabrielle and Jeff and I would like to welcome our third and last but not least is presenter one boolean, who is a second year MBA student from the Department of.
251
00:46:50.850 --> 00:47:04.860
SASHA RAZOR: Second language studies at the University of Hawaii at manoa USA her research interests include how people, adults and children mono lingual and bilingual process and learn language welcome one.
252
00:47:09.660 --> 00:47:10.980
Hoan Nguyen: me share my screen.
253
00:47:19.500 --> 00:47:20.850
Hoan Nguyen: Can you see my screen right now.
254
00:47:22.980 --> 00:47:23.400
Hoan Nguyen: Okay.
255
00:47:25.440 --> 00:47:39.630
Hoan Nguyen: I must say sorry in advance that the background, a little bit noise here, this is early morning, you might hear the birds singing or the car pacify or even the rooster and the chicken fighting in the chat so i'm really sorry about this.
256
00:47:41.190 --> 00:47:42.300
Hoan Nguyen: Okay, so.
257
00:47:43.860 --> 00:47:45.330
Hoan Nguyen: That representation.
258
00:47:48.930 --> 00:48:00.450
Hoan Nguyen: Hello everyone, my name is one and today i'm going to present my appreciate on developing and validating am listening proficiency test for enemies heritage runners.
259
00:48:03.540 --> 00:48:14.490
Hoan Nguyen: Just a brief introduction about the background of this study and what motivated me to develop this is research on heritage language learner show that.
260
00:48:15.390 --> 00:48:24.330
Hoan Nguyen: heritage language learner show strong in their skill over the other language skill according to carrier in cave in 2011.
261
00:48:24.870 --> 00:48:32.940
Hoan Nguyen: holiday speakers Center better at listening then they're speaking and their self report also emphasize literally reference.
262
00:48:33.690 --> 00:48:46.260
Hoan Nguyen: As as listening proficiency in my career Tom there with processing, for example, they are predicting an upcoming now, after hearing a classifier in Vietnamese or a gender marker event.
263
00:48:47.550 --> 00:48:49.800
Hoan Nguyen: However, as far as I know.
264
00:48:51.360 --> 00:48:56.700
Hoan Nguyen: Most of the heritage language learner assessment focus on the most commonly taught English.
265
00:48:57.180 --> 00:49:13.230
Hoan Nguyen: Yes, English, Spanish, Russia or yum and where's is no standard those long reference listening test for been a miss heritage learner share some commercial test but it's very expensive and it's not for the research proposal, so I cannot afford it.
266
00:49:14.250 --> 00:49:21.510
Hoan Nguyen: We should encourage me to live love this test as as a participant listening proficiency and uses as.
267
00:49:22.620 --> 00:49:25.140
Hoan Nguyen: In my language processing shy.
268
00:49:28.080 --> 00:49:33.540
Hoan Nguyen: To develop that death I follow the argument based approach to validate it.
269
00:49:35.760 --> 00:49:54.120
Hoan Nguyen: came 1992 which is you widely language testing so many study articles books and you just stations in the acumen base validity validity refers to the degree to which evidence and tra supports.
270
00:49:56.220 --> 00:50:11.310
Hoan Nguyen: Our interpretation of test one and the proposed us as well at least 2021 identify to fight for characteristic of a given race validity and validation.
271
00:50:12.120 --> 00:50:20.850
Hoan Nguyen: According to these otter validity is not a property of a test or test for that the scope interpretation and test you.
272
00:50:21.420 --> 00:50:39.570
Hoan Nguyen: The validity is in literary literary is validity is menteith basis that are multi dimensional alliteration require the integration of multiple types of evidence and lastly validation is an ongoing process with multiple stage and faces.
273
00:50:44.100 --> 00:51:00.780
Hoan Nguyen: And here's how the framework looks like it's the shame of different seven different influence operation realization evaluation generalization a speculation extrapolation utilization and impact.
274
00:51:01.440 --> 00:51:15.900
Hoan Nguyen: In the scope of this presentation, I only focus on the two main types of influence, which are evaluation and generalization and I will go every focus on like introduce operational ization approach.
275
00:51:18.120 --> 00:51:18.630
Hoan Nguyen: and
276
00:51:20.250 --> 00:51:27.660
Hoan Nguyen: This approach, provide a supportive framework to formulate the research question, and obviously study.
277
00:51:37.770 --> 00:51:39.750
Hoan Nguyen: So this study at to.
278
00:51:41.430 --> 00:51:51.600
Hoan Nguyen: Live love and validate the Vietnamese listening proficiency test which is designed to measure of enemies hesitation or languish as general initially ability follow the.
279
00:51:52.710 --> 00:52:11.130
Hoan Nguyen: Guidelines literally the desk phone will be used as a dependent variable why study on with processing, as I briefly mentioned before, I will focus on the two main influence it wish our evaluation and genuine right he under evaluation, I know what to.
280
00:52:12.480 --> 00:52:22.200
Hoan Nguyen: Do with such question the first one is to what extent do the test item function properly and perform as intended, and the second question was.
281
00:52:22.890 --> 00:52:38.580
Hoan Nguyen: there's a test or follow the predicted pattern among groups and the generalization inference Oliver look into the test for on the video music listening proficiency test to see if those are internally consistent.
282
00:52:43.590 --> 00:52:51.810
Hoan Nguyen: To find the answer for this question I Palo the test on 24 universities students at the University of Hawaii at manoa.
283
00:52:52.350 --> 00:53:00.540
Hoan Nguyen: 19 of them are venues hesitation owners and the rest of them are native Speaker of the nominees the criterion for the hesitate learner go.
284
00:53:01.410 --> 00:53:19.440
Hoan Nguyen: To be included in this study, they have to come from he families in the US, and they are attend or attend at least one William is called Jewish an atomic participate participation and then he might be 18 or older.
285
00:53:21.900 --> 00:53:24.150
Hoan Nguyen: And here's how I develop a test.
286
00:53:25.260 --> 00:53:33.540
Hoan Nguyen: First, I review the APP for proficiency guidelines for listening and for Kendall statement to the to my function is and the test contents.
287
00:53:34.110 --> 00:53:46.830
Hoan Nguyen: Then I need with language testing a spurt to discuss testing process cases for each level number two Tom purposes, and so the time of the tests and.
288
00:53:47.760 --> 00:53:58.050
Hoan Nguyen: Next I reviewed as the government little richer to the side of question and respond to the number of choices audio play terms as well as the language of instruction.
289
00:53:58.830 --> 00:54:11.280
Hoan Nguyen: And finally I assess the authentic listening material and design a test item on the self hosted open shark platform pass testing for the sake of time, I will only cover the first step.
290
00:54:12.390 --> 00:54:19.560
Hoan Nguyen: This also the illustration of the operational ization Israelis and the base to validity.
291
00:54:21.060 --> 00:54:40.290
Hoan Nguyen: As you can see, on the graph on the right side of the screen here is how the fall guy like a proficiency look like, as this guideline if I consider fine measure levels of proficiency novus hysteria offense superior and distinguish.
292
00:54:43.680 --> 00:54:52.170
Hoan Nguyen: ish measure level of proficiency and representative different range of that language learner abilities under the.
293
00:54:53.520 --> 00:55:02.820
Hoan Nguyen: Major levels like novice intermediate and advanced they have the same level includes high level medium and low level.
294
00:55:03.480 --> 00:55:09.660
Hoan Nguyen: In the scope of this study, I will only focus on the three levels from its immediate to superior.
295
00:55:10.470 --> 00:55:24.270
Hoan Nguyen: Based on the skyline I am first identify the proficiency go I went to measure identify the journey select a topic that a shy and the content ally, with the level from intermediate to superior.
296
00:55:25.170 --> 00:55:39.840
Hoan Nguyen: One of the reason that I select this proficiency range, because after researching the Burmese class which we're offering a tuition model at that time they were offering three class level beginner intermediate and.
297
00:55:41.310 --> 00:55:57.960
Hoan Nguyen: Advanced However, given the fact that hesitation owners grew up in a family's aware, at least one parent speaker enemies as dominant language at home and decide to inside a test status desperately intermediate low level instead of the novice level.
298
00:56:00.330 --> 00:56:17.160
Hoan Nguyen: And here's a test format, the test include three pass the live with three different levels are following the full by line with a total number of questions 24 and the participant will give him 25 minutes to complete the help section.
299
00:56:19.560 --> 00:56:29.100
Hoan Nguyen: This test is contracted a not using the cloud version of our testing, which is one of the famous investing platform for.
300
00:56:30.570 --> 00:56:49.650
Hoan Nguyen: For this study, you can see that, like on the right corner here is show the progress bar and the carve out time to have a test taker manisha progress, and then the display time of the oil the quad also control to so the listener can only listen to the recording once one time.
301
00:56:52.530 --> 00:57:18.090
Hoan Nguyen: For that analyzes under the evaluation influence research question the it difficulty or item facility, I asked and the item discrimination our idea was calculated for our generation research question the callbacks and flowers collect calculated following the formula implant fine.
302
00:57:19.980 --> 00:57:21.300
Hoan Nguyen: And here, come the result.
303
00:57:23.100 --> 00:57:39.900
Hoan Nguyen: You can see that table one represent the descriptive statistics of the music listening proficiency two girls, or you can see, the main or the test arm is 1725 the median is 18 and the motor is 19.
304
00:57:41.670 --> 00:57:59.340
Hoan Nguyen: With a negative go to CES with a value of negative one point 19 and slightly negative still you can see the just me, you can see, the spread of the desk or for all the participants in this graph.
305
00:58:05.670 --> 00:58:08.100
Hoan Nguyen: To check whether or not the item.
306
00:58:09.780 --> 00:58:12.360
Hoan Nguyen: function properly the item has led.
307
00:58:14.280 --> 00:58:15.270
Hoan Nguyen: was calculated.
308
00:58:16.350 --> 00:58:24.810
Hoan Nguyen: The range the range gives me the Venice from a negative point you need to hear oh.
309
00:58:29.130 --> 00:58:39.780
Hoan Nguyen: Oh i'm sorry the item facility, the readiness for my quite 29 which indicates that greatly difficulty to one which indicates easy and other.
310
00:58:41.250 --> 00:58:53.310
Hoan Nguyen: Law ID evaluation negative point you need to to apply 62 better understand item facility value I create a.
311
00:58:53.970 --> 00:59:05.550
Hoan Nguyen: graph showing arm and here's how the graphic like you can choose a spread of the item means i'm ideally that the test it should have.
312
00:59:06.420 --> 00:59:27.960
Hoan Nguyen: The if value a 4.32 point 70 However, for this study, because of the smaller sample test item, a small sample size, I plan to keep the item in the health of the value in the range form point can defy 2.7 define and I will keep monitoring this item.
313
00:59:29.190 --> 00:59:42.750
Hoan Nguyen: However, I will also go back and check some of the item like three or five where's all the participant answer it correctly, I might need to I might need to share it or maybe come up with more effective.
314
00:59:43.950 --> 00:59:59.700
Hoan Nguyen: extractor sure to share this item, and I will show go back and revise some of the items I have the negative ID value of that mean that may be something wrong with these items in the student hall have a toast for.
315
01:00:00.300 --> 01:00:11.580
Hoan Nguyen: Like higher to score answer this question incorrectly with a student who has lower total score of the test or answer this item correctly.
316
01:00:14.370 --> 01:00:15.480
Hoan Nguyen: And then, to see if.
317
01:00:16.560 --> 01:00:17.640
Hoan Nguyen: The best or.
318
01:00:18.750 --> 01:00:27.630
Hoan Nguyen: perform as attended a plus a box plot was generated on our which indicate group variation as eternity.
319
01:00:28.230 --> 01:00:44.730
Hoan Nguyen: Basically, was divided into four different goals one group of made the speaker this group I call them a spurt and the other three groups ally, with their class level that one one beginner intermediate and offense the results show that.
320
01:00:47.280 --> 01:00:51.330
Hoan Nguyen: The enemy some like beginner course has the lowest.
321
01:00:52.410 --> 01:00:54.660
Hoan Nguyen: item in the group means.
322
01:00:57.990 --> 01:01:06.240
Hoan Nguyen: And then, when the class level goes up the means to increase do you can see, the trend here right.
323
01:01:09.990 --> 01:01:15.840
Hoan Nguyen: yeah we should mean that it's there's some differences student very well in different.
324
01:01:16.980 --> 01:01:21.960
Hoan Nguyen: Levels ally, with their according to their prior efficiencies.
325
01:01:24.270 --> 01:01:31.950
Hoan Nguyen: And so the research question under the generalization difference, the chrome back and fire was calculated.
326
01:01:33.360 --> 01:01:48.660
Hoan Nguyen: You can see here the data shows that the value of the reliability value before twice 70 which indicated a good reliability, all that mean that test some leisure, or what is supposed to measure and then all the.
327
01:01:49.680 --> 01:01:52.080
Hoan Nguyen: Best item measure the same content.
328
01:01:54.960 --> 01:02:04.290
Hoan Nguyen: Okay, in conclusion, there is a validity evidence of what they use they need to be station of the test scores for the intended purpose.
329
01:02:04.950 --> 01:02:16.800
Hoan Nguyen: Although there are some limitation of the study study is dialect that the speaker have when recording the audios and then the small sample size.
330
01:02:17.310 --> 01:02:28.770
Hoan Nguyen: The study demonstrate a good model to develop other similar listening proficiency test for other heritage or less commonly tash languish using the apple proficiency guidelines.
331
01:02:30.300 --> 01:02:32.310
Hoan Nguyen: Is some relative preference.
332
01:02:33.480 --> 01:02:39.600
Hoan Nguyen: that's the end of my presentation, thank you for your listening and let me know if you have any questions, thank you.
333
01:02:41.250 --> 01:02:43.500
SASHA RAZOR: Thank you so much Horn.
334
01:02:43.560 --> 01:02:45.120
SASHA RAZOR: And congratulation.
335
01:02:45.240 --> 01:02:50.100
SASHA RAZOR: door congratulations to all presenters and doing very interesting presentations.
336
01:02:50.580 --> 01:02:59.970
SASHA RAZOR: And I am sure, because we have 17 participants in this panel i'm sure that lobby, a lot of questions that you can ask each other and specific questions.
337
01:03:00.270 --> 01:03:13.350
SASHA RAZOR: But before we go to them, I would like to ask a very general question I heard it mentioned a couple of times in the first two presentations that there is a great deal of conflation between your heritage speakers, and the second.
338
01:03:13.770 --> 01:03:24.030
SASHA RAZOR: Is heritage learners and second language learners and my question is, do you have a sense, based on your pedagogical practice that your heritage learners are changing.
339
01:03:24.600 --> 01:03:40.170
SASHA RAZOR: And if they do, what are you planning to do to adjust your tests, I don't know if this is true for the Spanish language programs, but we certainly experienced that in the Russian language programs some just maybe you could address it briefly.
340
01:03:42.600 --> 01:03:43.170
heritage.
341
01:03:44.670 --> 01:03:56.520
SASHA RAZOR: in regards to changing are you are you noticing noticing anything different about your heritage our heritage language proficiency levels fluctuates no not at all because it's certainly true for us.
342
01:03:56.790 --> 01:04:00.720
Jeffrey Longwell: Of course it is changing, simply because of the fluctuation and.
343
01:04:01.080 --> 01:04:06.090
Jeffrey Longwell: In our migrations we are close to the border we're about 35 minutes away from the border so.
344
01:04:06.330 --> 01:04:14.760
Jeffrey Longwell: anytime that we have a specific way wave of immigrants coming in and out of the country, it changes the the profile of of the type of student that we're getting.
345
01:04:15.030 --> 01:04:23.970
Jeffrey Longwell: In this particular case, new Mexico is very unique because it has a population that that runs from the northern part of the state to the southern part of the state which.
346
01:04:24.630 --> 01:04:35.610
Jeffrey Longwell: is also very diverse the northern parts tend to concentrate more on on a more colloquial Spanish if we can even tendencies to to have some some.
347
01:04:36.000 --> 01:04:42.780
Jeffrey Longwell: archaic Spanish and then, when we deal with the Mexican population coming in in the southern part they bring a lot of the.
348
01:04:43.260 --> 01:04:50.550
Jeffrey Longwell: dialects from the northern part of Mexico, so of course it keeps changing depending on the changes in our government and the changing changes in our.
349
01:04:51.060 --> 01:04:59.520
Jeffrey Longwell: migration patterns that we see throughout the state so for us, yes it is changing and the way we've been adapting is we're constantly testing our placement exam.
350
01:05:00.390 --> 01:05:09.960
SASHA RAZOR: But do you feel the need to alter this placement exam so little bit all the changes are not consistent, so there is no way to capture it through a little formula is it.
351
01:05:10.260 --> 01:05:23.850
Jeffrey Longwell: Well it's my own experience, because I am a heritage language learner myself and the way that i've seen it it's that it's changes that come every 10 years every 10 years we've had to adapt and change your exam that's exactly why we made the change from our previous exam.
352
01:05:24.120 --> 01:05:39.120
Jeffrey Longwell: And we're testing this one, because of the changes in our population So yes, I agree with you that those changes will come for me what i've experienced is every 10 years we've had to adapt to a different a different focus within our placement exam.
353
01:05:39.450 --> 01:05:47.190
SASHA RAZOR: yeah and and what does it for you, even a mother, because you work in California with you know also migration.
354
01:05:51.630 --> 01:05:52.200
Eve Zyzik: well.
355
01:05:53.220 --> 01:05:55.740
Eve Zyzik: I guess i'll i'll try to answer the question.
356
01:05:56.910 --> 01:06:09.540
Eve Zyzik: We, our goal is really to see if we can, if our test and we haven't gotten there really yet is if our test can provide different groupings within the heritage language learner population.
357
01:06:10.230 --> 01:06:17.790
Eve Zyzik: Think we're a little bit less concerned about making the distinction between L two and heritage, although.
358
01:06:18.330 --> 01:06:29.100
Eve Zyzik: that's a question and grant put all asked earlier in the chat whether we what would happen if we gave this test to lt learners but we really more interested in seeing if our test can.
359
01:06:30.330 --> 01:06:33.360
Eve Zyzik: be useful for both research and.
360
01:06:34.440 --> 01:06:48.930
Eve Zyzik: University placement in grouping heritage learners into let's say three different groups and i'm just throwing the number three out there for hypothetical imani I don't know if you want to follow up.
361
01:06:51.120 --> 01:07:00.000
Amàlia Llombart: I mean, since this is a new test we don't have a point of reference and the only thing would be like observation are like my impression in the classroom.
362
01:07:00.540 --> 01:07:10.080
Amàlia Llombart: Which is either not because of epidemic and the general like different attitude that i'm seeing in all my students in general.
363
01:07:10.800 --> 01:07:29.670
Amàlia Llombart: I it wouldn't be a good moment to make any judgment or any impressions, because there are no this is less year, for example, i've seen all my students in all my classes habit different I don't know if if I if I were to make a judgment no wouldn't be very bus.
364
01:07:31.200 --> 01:07:37.770
Amàlia Llombart: But, but I don't think it's relevant I mean it's really related to that, but, but now you're making me think.
365
01:07:38.790 --> 01:07:45.120
Amàlia Llombart: And now i'm going to look at that because I mean if you observe something and that might be something to see that.
366
01:07:46.590 --> 01:07:47.700
Amàlia Llombart: missed but.
367
01:07:48.300 --> 01:08:02.760
SASHA RAZOR: What What about the pandemic and the whole language transmission situation changes and one did you have anything to add to it, because I know that you've only worked with like 25 students but maybe you have prior experience teaching and observation.
368
01:08:03.960 --> 01:08:04.950
Hoan Nguyen: yeah so.
369
01:08:06.030 --> 01:08:23.760
Hoan Nguyen: The data I show in my presentation in the first layer so for five for my scholarly paper, I still use this test as a independent variable so I did include the outcome learners in my study and then the without the first result that I see is their.
370
01:08:25.530 --> 01:08:34.650
Hoan Nguyen: Their squad is significantly lower than the heritage group a given the fact like I said on the hesitation on a group have very.
371
01:08:35.310 --> 01:08:46.830
Hoan Nguyen: Strong listening school because they spoke to the language in their everyday life, compared to the Alto learners they only listen, or maybe have a chance to I spoke to the enemies.
372
01:08:47.580 --> 01:09:00.690
Hoan Nguyen: In their class in the university, so I can see that the data is spread a lot on the left side as me and I want to squat when we're able to illness involved in this Daddy.
373
01:09:01.950 --> 01:09:13.650
SASHA RAZOR: Thank you, thank you, so I would like to open the floor to questions from all participants and an alpha 14 on the panel and let's see what kind of questions come from the audience.
374
01:09:15.840 --> 01:09:21.510
SASHA RAZOR: So if is asking the question one, what is the web, plus from the to use for your test.
375
01:09:24.510 --> 01:09:32.160
Hoan Nguyen: Okay, thank you for your question, and the last one i'm using is about testing sorry for the Nice and.
376
01:09:32.910 --> 01:09:43.230
Hoan Nguyen: Unfortunately, and this platform only offer the package for like the big company or a situation, so in my case we've got a have some knowledge in like.
377
01:09:44.070 --> 01:09:59.610
Hoan Nguyen: web based and html so that's why I can install the with the half of my friend, I can eat all the core version doesn't mean the free the free version for individual so I can deliver the test and then houses on my own web based costing.
378
01:10:01.350 --> 01:10:05.460
Hoan Nguyen: So yeah the the name of the course is testing platform.
379
01:10:06.750 --> 01:10:08.370
Hoan Nguyen: Well, yes, very good forum.
380
01:10:08.670 --> 01:10:19.620
Hoan Nguyen: Like a participant less than 100 participants, but if more than one participant I might need to come up with different platform, or even though I don't think I can push a sub.
381
01:10:20.520 --> 01:10:29.130
Hoan Nguyen: The software because it's terribly expensive yeah, thank you for visiting the link on the chat house, you can refer to that one to see the platform I use.
382
01:10:30.540 --> 01:10:40.350
SASHA RAZOR: Thank you think and it's logical to ask what what are your testing platforms them, what about a new Mexico and Eve and Amalia.
383
01:10:41.610 --> 01:10:48.480
Jeffrey Longwell: So here in new Mexico, we are basically using a Google form.
384
01:10:49.860 --> 01:10:53.070
Jeffrey Longwell: We do not distribute the.
385
01:10:54.120 --> 01:11:01.380
Jeffrey Longwell: The link widely because we designed, which is why we're very protective of the test we we don't want.
386
01:11:03.330 --> 01:11:11.940
Jeffrey Longwell: Students outside of the university to use it, because then it would block our access to our students, because when it sends out I have it.
387
01:11:12.360 --> 01:11:20.490
Jeffrey Longwell: program to where it will calculate things based on thresholds that we did way back when we were piloting our original placement exam.
388
01:11:21.090 --> 01:11:30.900
Jeffrey Longwell: And so, based on the responses to certain things and the score levels that come up, it will calculate a placement recommendation.
389
01:11:31.380 --> 01:11:41.040
Jeffrey Longwell: And then it will send an email to the student advising them of the placement recommendation so that's why we, we only pretty much use it either.
390
01:11:41.400 --> 01:11:52.440
Jeffrey Longwell: Right at the beginning of the Semester within our courses, or when the advising is going on and we send that link directly to students that have an enemy Su email.
391
01:11:53.640 --> 01:12:06.180
Jeffrey Longwell: So we would love to have something that had you know, a nicer firewall attached to it, but again using simple, accessible things like Google and if you.
392
01:12:07.260 --> 01:12:16.620
Jeffrey Longwell: have just some very basic knowledge in terms of programming formulas and things if then statements embedded if then statements in.
393
01:12:17.040 --> 01:12:28.080
Jeffrey Longwell: excel, then you can you can actually calculate that out and send the the results out to the students and then it's also again it's great for doing.
394
01:12:28.740 --> 01:12:42.720
Jeffrey Longwell: excel is great for doing item statistics spss is not that great doing item statistics it's great for doing a novus it's great for doing other things, but it's not great, for doing it and statistics yeah.
395
01:12:44.790 --> 01:12:47.760
SASHA RAZOR: All right, and what about even amelia.
396
01:12:50.460 --> 01:12:54.210
Eve Zyzik: We have ours is in call tricks okay cool tricks.
397
01:12:55.980 --> 01:12:57.000
SASHA RAZOR: Have you tried different.
398
01:12:57.000 --> 01:12:58.470
SASHA RAZOR: desktop platforms, as you will.
399
01:12:58.470 --> 01:13:01.230
SASHA RAZOR: Work no you just select that one, and that was it.
400
01:13:01.590 --> 01:13:02.910
Eve Zyzik: No, we just selected.
401
01:13:02.910 --> 01:13:09.480
Eve Zyzik: QUAL tricks, I mean ours is really simple in terms of the format, we don't have any audio or anything like this.
402
01:13:09.960 --> 01:13:11.160
Eve Zyzik: But we could add it.
403
01:13:11.970 --> 01:13:29.490
Eve Zyzik: But you know it's just that they see the word they see four options, plus the five options, including the I don't know, and I think amaya how many do we have on a screen at once, like five or something at once, so it doesn't get too overwhelming yeah.
404
01:13:31.170 --> 01:13:33.150
Eve Zyzik: it's called tricks works great for.
405
01:13:33.210 --> 01:13:34.320
Our purposes.
406
01:13:35.460 --> 01:13:37.560
Hoan Nguyen: yeah I didn't know that the question to.
407
01:13:37.800 --> 01:13:54.270
Hoan Nguyen: That like you know is what very well for the vocabulary tests before listening test run I need to control the display Tom that the recording time so I need to look as yeah to see other platform and to by mass function, and then on.
408
01:13:55.290 --> 01:14:05.370
Hoan Nguyen: Thursday I think they also provide the like score automatically you can set the correct answer, and then it will automatically add to.
409
01:14:06.990 --> 01:14:07.620
This one this year.
410
01:14:08.670 --> 01:14:18.630
SASHA RAZOR: Which is really great and i'm going to ask one more really selfish question because I mean this term does for high school students have your programs.
411
01:14:18.660 --> 01:14:19.920
SASHA RAZOR: Try to work with.
412
01:14:20.100 --> 01:14:24.990
SASHA RAZOR: heritage learners and high schools and If yes What was your experience.
413
01:14:29.370 --> 01:14:42.270
Jeffrey Longwell: By as far as our program we do offer what's called a dual program so we do have high school students coming in to the two enemies huge main campus to start taking their classes on our campus.
414
01:14:42.600 --> 01:14:44.460
Jeffrey Longwell: us taking the exam to.
415
01:14:45.150 --> 01:14:53.550
Jeffrey Longwell: High School we haven't done that, but our population is coming locally, so the are coming from the local high schools and the testings to be working.
416
01:14:53.820 --> 01:14:56.130
Jeffrey Longwell: effectively with that population as well.
417
01:14:56.670 --> 01:15:05.070
Jeffrey Longwell: Going back to your previous question we've also struggled with finding a place to to put our bar exam because, again, we want to remain.
418
01:15:05.400 --> 01:15:14.280
Jeffrey Longwell: loyal to our students and make sure that it's that it's test given only to animus your students and, of course, this is all done with no money so all our work.
419
01:15:14.940 --> 01:15:27.960
Jeffrey Longwell: was done trying to find a platform that's free that will allow us to do that, but i'm just to go back to both questions we continue to work with that population we're recruiting from that high school population and the exam seems to be doing well.
420
01:15:29.430 --> 01:15:45.300
SASHA RAZOR: This is this is great to know and it's really inspiring to see the this knowledge is really transferable to other languages and might be developed and applied for other and I guess the last question that I have if we do not get more questions.
421
01:15:45.300 --> 01:15:46.500
SASHA RAZOR: From the audience is.
422
01:15:46.740 --> 01:15:49.050
SASHA RAZOR: What are your next steps in developing your.
423
01:15:49.050 --> 01:15:53.550
SASHA RAZOR: desk placement us and you're destined platforms like, how do you see the development.
424
01:15:53.610 --> 01:15:54.420
SASHA RAZOR: what's next.
425
01:15:59.070 --> 01:16:22.470
Jeffrey Longwell: Well, I can, I can say that this was sort of our test for adding new items in because a good thing in terms of monitoring the test and making sure that the test is somewhat secure, because at any given point somebody could share the answers and just bring years of work to the ground.
426
01:16:23.490 --> 01:16:29.130
Jeffrey Longwell: So we like to have new items that we can rotate through and keep things fresh and.
427
01:16:29.130 --> 01:16:30.510
Amàlia Llombart: Make sure that that.
428
01:16:30.540 --> 01:16:31.350
Jeffrey Longwell: We are.
429
01:16:31.590 --> 01:16:44.070
Jeffrey Longwell: Still aligned with what who our student population is and what the the different course levels are that we have so it's continuing to do that and then.
430
01:16:44.580 --> 01:16:50.430
Jeffrey Longwell: Some of the things that we mentioned that that Jeff mentioned is that's our next step is.
431
01:16:52.020 --> 01:16:52.590
Jeffrey Longwell: going out.
432
01:16:52.650 --> 01:17:07.770
Jeffrey Longwell: what's what's happening with those learners that are we know we've identified them as heritage learners but who's picking the second language they stay in the second language classes what what's the motivation for doing that and what's the outcome does if.
433
01:17:08.850 --> 01:17:14.820
Jeffrey Longwell: For our heritage we separated people out and you saw that our heritage learners that we're in our.
434
01:17:15.510 --> 01:17:30.930
Jeffrey Longwell: L two classes, they performed better than the l twos and the lt classes, but they performed less well than the heritage learners enrolled in the heritage classes, so we want to know, are these characteristics that are.
435
01:17:31.560 --> 01:17:44.730
Jeffrey Longwell: You know, related to ability levels and and learners that feel that they don't have the ability to be in a heritage course are they self selecting and staying in that or.
436
01:17:45.840 --> 01:17:49.800
Jeffrey Longwell: Is it that the experience that they're having that there.
437
01:17:49.830 --> 01:17:51.750
Jeffrey Longwell: there's some conflict where they're.
438
01:17:51.750 --> 01:18:03.690
Jeffrey Longwell: questioning their own abilities, because they're learning specific rules that that conflict with the way that they have language organized in their brain so.
439
01:18:04.080 --> 01:18:15.000
Jeffrey Longwell: Is it that conflict that's influencing are they are they doubting do I notice joy I don't know that you know things like that so Those are the things that we're looking to tease out a little bit more.
440
01:18:15.090 --> 01:18:18.720
SASHA RAZOR: Thank you for that so done the resonates with me as a language teacher and.
441
01:18:18.750 --> 01:18:20.760
SASHA RAZOR: Mr Leo did you have to add something.
442
01:18:21.810 --> 01:18:25.380
Amàlia Llombart: I totally respect him before because I wanted to ask a question so.
443
01:18:26.280 --> 01:18:26.700
Amàlia Llombart: I can.
444
01:18:27.570 --> 01:18:34.620
Amàlia Llombart: Well, I just want to ask the question that grant as before and put it on the on the chat, but I think you know.
445
01:18:35.490 --> 01:18:48.540
Amàlia Llombart: Which is that what i'm going to reward said but i'm curious when you mentioned satisfied items and subjective items, what exactly does this look like and i'm curious if you can see just one I mean I promise I won't.
446
01:18:48.570 --> 01:18:48.930
Tell me.
447
01:18:50.220 --> 01:18:51.660
you're all sworn to secrecy.
448
01:18:56.010 --> 01:18:56.910
Jeffrey Longwell: that's fine we know.
449
01:18:56.970 --> 01:18:58.350
Amàlia Llombart: We mix up all of.
450
01:18:59.430 --> 01:19:16.770
Jeffrey Longwell: them yeah we We trust you all um so we're we're looking at canonical uses of sadness that we're we're looking at things like my idea, and then a blank I can fix that so okay very simple things like that we give them the four.
451
01:19:17.010 --> 01:19:28.320
Jeffrey Longwell: Four options and then the no settle other sports that at the end I don't know the answer and and what we do is and i'm sure it was on the slide the.
452
01:19:28.770 --> 01:19:34.500
Jeffrey Longwell: All of those options are randomized so the orders, the questions are ordered.
453
01:19:35.100 --> 01:19:42.540
Jeffrey Longwell: It they appear in the same order, but the options for each question every time they pop up their their randomized for different learners.
454
01:19:43.050 --> 01:19:54.420
Jeffrey Longwell: So the know seven letters, first, I mean we programmed it in at the bottom of each, but it may pop up at the beginning, or in the middle, depending on who's teaching or who's taking the test at any given time.
455
01:19:55.800 --> 01:20:03.750
Jeffrey Longwell: So that's one of the things that we think is is helping to say people aren't just guessing they're really having to consider all of the options.
456
01:20:06.180 --> 01:20:11.340
Jeffrey Longwell: we're not we're not using items that are that can change, for instance, if.
457
01:20:11.340 --> 01:20:12.540
Jeffrey Longwell: You say it's Alto.
458
01:20:12.600 --> 01:20:13.710
Amàlia Llombart: or Estado.
459
01:20:14.160 --> 01:20:14.760
Jeffrey Longwell: You know that.
460
01:20:14.790 --> 01:20:24.150
Jeffrey Longwell: That is based on people's perception right so we're not looking at things like that we're using things like a snack on saw that she's tired or something.
461
01:20:24.540 --> 01:20:33.660
Jeffrey Longwell: So we're looking at very canonical uses of setting aside and with a substantive we've included different varieties of subjunctive because.
462
01:20:35.460 --> 01:20:48.960
Jeffrey Longwell: heritage language learners learn it's subjective is learned over time right and it's learned in different ways, so pretty much the baseline that even like three year old kids know is you know the the.
463
01:20:48.990 --> 01:20:50.010
Amàlia Llombart: verbs of volition.
464
01:20:50.640 --> 01:21:01.950
Jeffrey Longwell: right because that's what you hear a lot, I want you to do this sort of thing so that gets that gets embedded in there, so that's sort of the baseline and then we move on from there when.
465
01:21:03.570 --> 01:21:10.230
Jeffrey Longwell: we've got some that are we've got two I think in there that are everything's in present tense.
466
01:21:11.190 --> 01:21:18.360
Jeffrey Longwell: we've got two that are see clauses and, of course, those are in the past tense because that's just how.
467
01:21:18.660 --> 01:21:33.630
Jeffrey Longwell: The languages is structured, but everything else is in present tense we kind of try try to get things as bare bones as possible, just to make it simple just to make it easy not to have any of you know that.
468
01:21:35.910 --> 01:21:36.630
Jeffrey Longwell: Cut that.
469
01:21:36.840 --> 01:21:37.530
Jeffrey Longwell: what's it called.
470
01:21:37.560 --> 01:21:47.310
Jeffrey Longwell: A constructor relevant difficulty that messick talks about where it's these unnecessary hurdles that you make learners go through just to find.
471
01:21:47.730 --> 01:21:58.770
Jeffrey Longwell: To get to show their knowledge to display their their knowledge about what what the right question is so we we try to keep things as clean as possible as simple as possible.
472
01:21:59.310 --> 01:22:20.670
Jeffrey Longwell: And again what we found our that we have these amazing stair step outcomes on these very simple items which we're we're thrilled about because honestly we had previously a test that was 100 items long, and when I did about.
473
01:22:21.720 --> 01:22:36.600
Jeffrey Longwell: 1012 years ago an item analysis on our heritage learners based on that test it was embarrassing because none of the items were really doing what they should be doing so, we were.
474
01:22:37.290 --> 01:22:54.390
Jeffrey Longwell: torturing and I it's still it's still a dagger in my heart, because I think we we used I when I came my the test was already here, I and they put me in charge of that test, but it wasn't for a few years later, until I actually did the item analyses and found that out.
475
01:22:55.620 --> 01:23:10.050
Jeffrey Longwell: That we had put students through this torture of displaying this knowledge for no good reason it was doing nothing, we could have thrown a dart at a wall and sit take that course, so this is my penance.
476
01:23:12.360 --> 01:23:33.060
Jeffrey Longwell: Trying to make sure that we are doing right by our learners and particularly as Gabriella wissing we're focusing on our heritage learners first on on finding what their skills and abilities are that's that's how we designed the.
477
01:23:33.120 --> 01:23:33.780
Amàlia Llombart: The items.
478
01:23:33.840 --> 01:23:48.300
Jeffrey Longwell: And then Lo and behold, it works with the lt learners to that's the that's The wonderful thing just just to add to that I wanted to address Dr seismic comment about her exam ours has been so beneficial in the.
479
01:23:48.300 --> 01:23:48.900
Jeffrey Longwell: sensitive.
480
01:23:48.960 --> 01:23:50.160
Jeffrey Longwell: been able to target.
481
01:23:50.220 --> 01:24:02.700
Jeffrey Longwell: native speakers heritage Center nurse and second language learners it's very user friendly it takes between 15 minutes to 30 minutes and, ultimately, our goal is just to play students, we don't want to know if they have.
482
01:24:03.240 --> 01:24:13.410
Jeffrey Longwell: Mental linguistic knowledge that's not the whole purpose of our test our test is where can we place our students, how can we help them guide them through this process in registering for our courses.
483
01:24:14.070 --> 01:24:24.390
Jeffrey Longwell: it's like I said very user friendly and it's very they're very comfortable even students are surprised at the end because we leave a blank for comments they'll stay, I thought that the test would be more difficult.
484
01:24:24.660 --> 01:24:29.970
Jeffrey Longwell: You know I find that I did really well, so it kind of enhances what they're bringing into the classroom.
485
01:24:30.540 --> 01:24:34.740
Jeffrey Longwell: We have published extensively on our placement exam not only the.
486
01:24:34.890 --> 01:24:37.260
Jeffrey Longwell: Set aside if you're interested in McGregor.
487
01:24:37.380 --> 01:24:44.610
Jeffrey Longwell: mode I know long well you know you can look up all that information and we're continuing on we have something coming up here with a focused on l too.
488
01:24:45.000 --> 01:24:57.360
Jeffrey Longwell: So definitely please look for our articles out there reach out to us, we are here to to help guide whatever we can because we've it's been a long process and we've worked our synergy is really, really.
489
01:24:57.960 --> 01:25:09.210
Jeffrey Longwell: Great and we've been able to accomplish a lot with this with this exam and and we really encourage everybody to just make their own tests, really.
490
01:25:09.270 --> 01:25:10.170
Jeffrey Longwell: To to just.
491
01:25:11.100 --> 01:25:29.130
Jeffrey Longwell: You know, build capacity to make your own test based on who the learners are that you're seeing coming into your program and and what your program what the goals of your program are and that's all we're doing is we're just as Gabriella was saying we're just looking to say where.
492
01:25:29.610 --> 01:25:30.480
Jeffrey Longwell: Where do we put you.
493
01:25:30.900 --> 01:25:34.260
Jeffrey Longwell: You know where do you fit you know it's like those when you go to the.
494
01:25:35.280 --> 01:25:36.510
Jeffrey Longwell: amusement park and it said you.
495
01:25:36.510 --> 01:25:36.900
Amàlia Llombart: must be.
496
01:25:36.930 --> 01:25:39.600
Jeffrey Longwell: This tall to ride this ride that's all we're doing.
497
01:25:40.140 --> 01:25:46.860
Jeffrey Longwell: Nothing fancy we're just saying how tall, are you based on where where are different levels are in our in our Program.
498
01:25:47.280 --> 01:25:59.160
Jeffrey Longwell: And sorry I keep interrupting but, most importantly, always remember that the skills are not always academic they're not always grammar it's not always Meta grammar, we have to look at the skills that students are bringing into.
499
01:25:59.370 --> 01:26:03.180
Jeffrey Longwell: funds of knowledge and we're playing with all of that inner inner.
500
01:26:03.570 --> 01:26:09.780
Jeffrey Longwell: placement exam you have to remember that bit, especially because of the of the question of how it's changing.
501
01:26:09.990 --> 01:26:16.290
Jeffrey Longwell: We have to start learning that the skills that they're bringing in have to be part of these exams, in order for us to be able to gauge.
502
01:26:16.590 --> 01:26:32.880
Jeffrey Longwell: What they're bringing into the classroom yeah so hearing he just hearing how your students are learning what are using language I mean i'm surprised, all the time hearing new slang coming in right you're great at picking up that a lot so just wow it's it's amazing.
503
01:26:34.920 --> 01:26:36.810
Jeffrey Longwell: Thank you everyone for your questions.
504
01:26:37.800 --> 01:26:43.590
SASHA RAZOR: Your, I would like to welcome the last opportunity to exchange something.
505
01:26:44.970 --> 01:26:49.260
SASHA RAZOR: ask questions etc does anyone else have any comments or questions.
506
01:26:51.510 --> 01:26:51.900
SASHA RAZOR: Please.
507
01:26:52.260 --> 01:27:04.560
Eve Zyzik: I would just jump in Thank you the team from new Mexico state it's really great to see you at physically in the same room and thing how much you enjoy working together.
508
01:27:04.890 --> 01:27:18.570
Eve Zyzik: I feel like you know, making your own test is daunting, but when you have somebody else that you're working it's really, really important to have at least one other person that you're working on these items with because, like.
509
01:27:19.500 --> 01:27:24.720
Eve Zyzik: A Monday and I were going through our test, and we were eliminating coordinates and it was amazing like.
510
01:27:26.310 --> 01:27:43.080
Eve Zyzik: You know it's it's not that we were disagreeing but she was picking up things saying this is a combination oh yes you're right absolutely it's a coordinate and vice versa right so when you're working with someone else it really gives you that opportunity to.
511
01:27:44.160 --> 01:27:53.730
Eve Zyzik: Make sure you're not the only voice in the room, yes um and when we're writing our definitions, you know same thing, there were words that I was struggling to define.
512
01:27:54.420 --> 01:28:10.080
Eve Zyzik: Not because I don't know what they mean, but I was struggling to define them in a kind of user friendly way, and she would just give a great definition and on we would go so yeah I think a really important the collaboration.
513
01:28:10.620 --> 01:28:21.870
Amàlia Llombart: It was funny when I was listening to Patricia before seen like the bare bones and cleaning because we use that that express it a lot, we still need to clean the definitions more like.
514
01:28:22.980 --> 01:28:24.630
Amàlia Llombart: pick up to be shorter than they.
515
01:28:24.780 --> 01:28:34.320
Amàlia Llombart: need to read all these shorter just more clear, so that, and it was very because you were expressing the same the same a BF what we want to convey with.
516
01:28:35.640 --> 01:28:56.100
Jeffrey Longwell: This test yeah, the main thing is, is that again going away from from what the old test was where you know it was just like I, I think that probably the students were traumatized an emotionally and cognitively exhausted, at the end of 100 question test.
517
01:28:57.150 --> 01:29:02.070
Jeffrey Longwell: yeah that just shine this light in every dark cobwebby corner of their.
518
01:29:02.340 --> 01:29:14.340
Jeffrey Longwell: of their linguistic knowledge and you had there was no sense of we're starting here we're ending there, it was just sort of bouncing around and I just felt, you know horrible for the students that were put through that so.
519
01:29:14.850 --> 01:29:26.610
Jeffrey Longwell: You know, finding things that are user friendly and Gabriella is she's brilliant at designing the the questions that are the the.
520
01:29:27.120 --> 01:29:35.280
Jeffrey Longwell: Non traditional readings just the the social media exchanges, because this is the way that our learners interact with each other all the time.
521
01:29:35.580 --> 01:29:47.970
Jeffrey Longwell: And so, using that as it as a way to index their literacy skills, rather than saying read this short passage on some topic that you may have zero interest in and will tell you if you know how to read or not.
522
01:29:49.110 --> 01:30:08.730
Jeffrey Longwell: You know just the incongruity of that and and again we're trying to do say you know where should we put you we will teach you, but we need to know what what you're bringing so that we can place you well, so that we can you know elevate you from where you are.
523
01:30:11.370 --> 01:30:29.280
SASHA RAZOR: This is wonderful, and it makes me think that there's 10 as an evolving process and not only that your learner's fluctuated change, you also have adopted this, you also have to refine your own methods, all the time, and all these things are definitely ECHO and with other language programs.
524
01:30:29.280 --> 01:30:34.140
SASHA RAZOR: And the skills, the skill sets a highly transferable and you can map them onto.
525
01:30:34.170 --> 01:30:35.520
SASHA RAZOR: Other programs so.
526
01:30:35.550 --> 01:30:47.970
SASHA RAZOR: I would like to thank everyone for participating today, and this is the very end of our Conference so well done, thank you very much and hope to see you last here next here okay.
527
01:30:48.450 --> 01:30:48.960
Eve Zyzik: Thank you.
528
01:30:52.140 --> 01:30:52.590
Jeffrey Longwell: bye.