Rapid Technological Change

UKLA O ccasio n al P ap er The \f nit e d K in gd om L it e ra cy A sso cia tio n M ult i m od a li ty b y E ve B earn e a n d K ath arin e V alle ly Mult i m od a li ty I n tr o d u \ftio n Rap id t e ch nolo gic al c h an ge h as m ean t t h at e ve ry d ay p ra ctic e s s u rro u ndin g r e ad in g a n d w rit in g h ave sh if t e d s ig n if ic an tly o ve r t h e la st 2 \b y e ars . M ost h om es h ave a cce ss t o a r a n ge o f d ig it a l d evic e s a n d ch ild re n a re b ro u gh t u p in a w orld w here t h ey e xp ect s c re en s t o b e t h eir c o m pan io n s, t o ys, e n te rta in ers , in fo rm atio n g iv e rs a n d m ean s o f s o cia l c o m munic atio n . M an y s c re en �b ase d t e xts c o m bin e w ord s w it h m ovin g im ag es, s o u nd, c o lo u r, a r a n ge o f p h oto gra p h ic , d ra w n o r d ig it a lly c re ate d v is u als ; s o m e a re in te ra ctiv e , e n co u ra g in g t h e r e ad er t o c o m pose , r e p re se n t a n d c o m munic ate t h ro u gh t h e s e ve ra l d im en sio n s o ffe re d b y t h e t e ch nolo gy. N ot o n ly a re t h ere n ew t y p es o f d ig it a l t e xts , h ow eve r, b u t a m assiv e p ro li f e ra tio n o f b o ok a n d m ag azin e t e xts w hic h u se im ag e, w ord , la yo u t a n d t y p ogra p h y, o ft e n ech oin g t h e d im en sio n s o f s c re en �b ase d t e ch nolo gy. T he a va ila b ili t y a n d f a m ili a rit y o f t h ese t e xts m ean t h at y o u ng p eo p le b rin g w id er e xp erie n ce o f t e xt t o t h e c la ssro om . B ut a lt h ou gh m an y c h ild re n exp erie n ce d ig it a l t e xts a n d e n vir o n m en ts f r o m a v e ry e arly a g e t h eir a cce ss t o t e ch nolo gy v a rie s co n sid era b ly . T here w ill b e d if fe re n ce s a cco rd in g t o s o cia l, c u lt u ra l, p ers o n al a n d e co n om ic f a cto rs s o th at in t h in kin g o f h ow b est t e ach ers c an r e sp on d t o t h e d ig it a l e xp erie n ce o f t h eir c h ild re n , f o cu sin g on s k ills is n ot e n ou gh . F ig u re s 1 a n d 2 C hild re n o f t h e s c re en a ge 2 M ult i m od a li ty b y E ve B earn e a n d K ath arin e V alle ly I t i s n ot on ly t h e t ext s a n d t h ei r t a ke� u p i n h omes a n d s c h ools w h i c h h a ve c h a n ged ; t h e w a ys i n w h i c h t h ey a r e c ompos ed , pr od u c ed a n d d i s t r i b u t ed h a ve u n d er gon e a fu n d a men t a l c h a n ge. New t ypes of t ext a n d , i n pa rt i c u la r , t h e d omi n a n c e of t h e s c r een , r eq u i r e c h a n ges i n la n gu a ge t o d es c r i b e t h em. I t ma y b e t h a t ‘ li t er a c y Õ or t h e s c h ool s u b j ec t ‘ En gli s h Õ a r e n ot s u ffi c i en t t o d es c r i b e 21s t c en t u ry t ext s , c on t ext s a n d pr a c t i c es : t h e c omb i n a t i on of t h e r epr es en t a t i on a l mod es of s peec h a n d s ou n d , w r i t i n g, im ag e , g e s ture , with the co m m unicative m e dia o f bo o k, m ag azine , co m p ute r s cre e n, vide o , f il m , radio . T his me ans that ap p roache s to te aching ne e d to be re s hap e d to accommodate to s hif ts in communica � tion a n d t ext exper i en c e i n a mu lt i mod a l, mu lt i med i a w or ld . O f c ou r s e, mu lt i mod a li t y i s n ot n ew . F or ma n y yea r s people of a ll c u lt u r es h a ve u s ed a r a n ge of mod es t o r epr es en t i d ea s a n d c ommu n i c a t e mea n i n g t h r ou gh s peec h , w r i t i n g, ges t u r e, i ma ge, movemen t , mu s i c a n d s ou n d . Even t h e mos t fa mi li a r a n d every d a y c ommu n i c a t i on s a r e ma d e u p of c omplex c omb i n a t i on s of mod es . Ta lk , w h et h er i n fa c e� t o� fa c e meet i n gs or vi ew ed on s c r een , i s a c c ompa n i ed b y movemen t a n d ges t u r e; t elevi s i on a n d fi lm c omb i n e movi n g i ma ges a n d s ou n d t o c r ea t e a n over a ll t ext . A n y mu lt i mod a l t ext mi gh t c omb i n e elemen t s of: •p e r f or ma n ce : ge s tu r e , move me n t, p os tu r e , f a ci a l e xp r e s s i on •i ma ge s : movi n g a n d s ti ll; p hotogr a p hi c, d r a wn , p a i n te d , comp u te r � ge n e r a te d e tc. •s ou n d : s p oke n wor d s , s ou n d e f f e cts , mu s i c a n d s i le n ce •wr i ti n g: i n clu d i n g f on t, gr a p hi cs a n d la you t •d u r a ti on : s hot le n gth, s e q u e n ce , r hythm a n d tr a n s i ti on s . (Bea r ne a nd Ba za l g ette, 2\b1\b:7) F i g u res \b an d 4 A mu l ti mod al project i n v ol v i n g d rama from T he T su n ami project: w r iting na r r a tive poe try b y Ja ne Be dna l l an d S u k wi n d er Bhod ay 3 Mult i m od a li ty Th es e elemen t s w i ll b e d i ffer en t ly w ei gh t ed i n a n y c omb i n a t i on of mod es , for exa mple, mi me h a s n o ve rbal dial o g ue , a p ho to g rap h is a s til l im ag e and a p ie ce o f s tre am e d ins tr um e ntal m us ic is jus t s o und, b u t fi lm i n c lu d es a ll t h e mod es : performa n c e, i ma ge, s ou n d , movemen t , ges t u r e, w or d s a n d d u r a t i on . Th e mu lt i mod a l s h i f t offer s s i gn i fi c a n t c h a llen ges t o t r a d i t i on a l forms of En gli s h / li t er a c y ed u c a t i on s o t h a t i n t h e c la s s r oom, c h i ld r en n eed t o b e h elped t o i d en t i f y t h e c on t r i b u t i on s ma d e b y ea c h mod e t o a n y t ext . I n a d d i t i on t o u s i n g d i ffer en t mod es t o r epr es en t a n d c ommu n i c a t e i d ea s , mes s a ges a r e r ela yed a n d d i s t r i b u t ed t h r ou gh d i ffer en t med i a of c ommu n i c a t i on . Th es e mi gh t b e t r a d i t i on a l med i a s u c h a s b ook s , gr a ph i c n ovels , ma ga z i n es a n d lea flet s , for exa mple, b u t d i gi t a l t ec h n ology n ow offer s t h e med i u m of t h e s c r een i n i t s va r i ou s forms a s w ell a s s ou n d fi les . C h i ld r en n ow t ype a s w ell a s w r i t e b y h a n d a n d u se sc r een n a v i ga t i on a l fa c i li t i es; m ob i le ph on es t r a n sm i t i m a ges a s w ell a s w or d s, c a n b e u sed t o pla y ga mes , a c c es s t h e i n t ern et a n d mu c h mor e � t h e n u mb er of a ppli c a t i on s i s q u i c k ly expa n d i n g. Af f ordan\fe s of mode s Th e mu lt i d i men s i on a l t ext s t h a t c h i ld r en a r e fa mi li a r w i t h a r e c on s t ru c t ed i n d i ffer en t w a ys fr om pr i n t t ext s a n d s o h a ve d i ffer en t a ffor d a n c es a n d d ema n d d i ffer en t w a ys of ma k i n g s en s e of t h em. I n t erms of c ommu n i c a t i n g mea n i n g, a ffor d a n c e i s a b ou t w h a t i t i s pos s i b le t o d o w i t h on e mod e or a n ot h er , or w i t h a c omb i n a t i on of mod es . Get t i n g a mes s a ge a c r os s c lea r ly a n d effec t i vely mea n s c h oos i n g c omb i n a t i on s of mod e( s ) a n d med i a a c c or d i n g t o w h a t t h ey a ffor d , or ma ke pos s i b le. Some mes s a ges a re b es t c a r ried b y wo rds s p o ken f a c e to f a c e; o thers m ig ht b es t b e written s o tha t they ha v e m o re p er m a nenc e a nd c a n b e re�rea d. W hen it c o m es to rea ding , a p rinted b o o k a f f o rds a dif f erent kind o f exp erienc e c o m p a red with wa tc hing a text o n s c reen. With a b o o k, the rea der c a n dec ide to s kip des c rip tiv e p a s s a g es , v a r y the p a c e o f rea ding a nd retur n to ea rl ier p a g es to c hec k deta il s o r rec a p ture the na r ra tiv e f l o w. Al tho ug h with rec o rding dev ic es there is a rev iew a nd f a s t f o r wa rd f a c il ity, ‘s kip p ing Õ p a r ts o f a udio o r v is ua l texts is ra ther m o re dif f ic ul t (a nd dis r up ts na r ra tiv e m ea ning ra ther m o re). With v is ua l texts , des c rip tiv e deta il is p a r t o f the im a g e s o tha t it is a l m o s t im p o s s ib l e to s kip o r ig no re el em ents . T he a f f o rda nc es o f f ered b y dif f erent m o des a nd m edia , then, inf l uenc e the wa y texts a re us ed, retur ned to , re�v iewed o r re�rea d a nd ho w they a re o rg a nis ed a nd c o ns tr uc ted a nd the c ho ic es m a de b y the rea der o r c o m p o s er/ writer. T ext \fo h esi o n Dif f er ent t yp es o f t ext ha ve va r ying o r g a nisa t io na l st r uc t ur es, o r p a t t er ns o f c o hesio n. Text s a r e sha p ed b y wha t the c o m b ina tio n o f m o des a nd m edia a f f o rd to the rea der o r v iewer: •W ritten na r ra tiv e o r rep o r t dep end o n c hro no l o g ic a l c o hes io n s o tha t idea s wil l b e l inked b y tim e c o nnec tiv es , f o r exa m p l e, then, l a ter, f ina l l y … •Texts whic h a re c o ns tr uc ted v is ua l l y o r dia g ra m m a tic a l l y dep end o n s p a tia l c o hes io n, us ing v is ua l links, for e xample , arrows or simply the j ux tapos itions of block s of pr int and pictur e s or diagr am s . •Te xts re laye d through the me dium of s ound, the s ingle v oice of a r adio ne w s r e ade r , for e x am ple , also de pe nd on chronological logic but in addition ar e m ade cohe s iv e by r e pe titions w hich w ould be re dundant in writte n te xts. •Te xts which are re laye d through physical m ov e m e nt, s ound and ge s tur e , for e x am ple , dr am a, dance , ope ra, imaginative play, combine both s patial and s ound�r e pe titiv e cohe s iv e de v ice s but in this case , the spatial is thre e �dime nsional. Young pe ople Õs te xt e xpe rie nce provide s the m w ith im plicit aw ar e ne s s that it is pos s ible to com bine diffe re nt mode s and me dia to ge t a me ss age acr os s . \f Mult i m od a li ty Re ading multimodal te xts I n t erms of pr i n t t ext s , ma n y b ook s a va i la b le i n s c h ools c a n n ot b e r ea d b y a t t en t i on t o w r i t i n g a lon e, a s mea n i n g a n d i n forma t i on a r e fr eq u en t ly c a rr i ed b y i ma ges , of t en pr es en t ed i n d ou b le pa ge s pr ea d s w h i c h a r e d es i gn ed t o u s e la you t , fon t s i z e, s h a pe a n d c olou r t o c omplemen t t h e i n forma t i on c a rr i ed b y t h e w or d s . C omplex mu lt i mod a l b ook s or ma ga z i n es u s e d i ffer en t r ea d i n g pa t h w a ys fr om r ea d i n g li n ea r n a rr a t i ve. Gu n t h er K r es s d es c r i b es t h e d i ffer en c es i n r ea d i n g pa t h w a ys b et w een d es i gn ed or d i s pla yed t ext a n d c on t i n u ou s pr i n t a s t h e d i ffer en c e b et w een t elli n g a n d s h ow i n g ( K r es s , 2\b\b3: 152) . I n r ea d i n g a pi ec e of c on t i n u ou s pr i n t ed t ext , a s t ory, for exa mple, t h e r ea d er i s t old a b ou t t h e r ela t i on s h i ps b et w een even t s t h r ou gh a s eq u en c e of s en t en c es ; i n r ea d i n g a n i ma ge, t h e r ea d er i s s h ow n r ela t i on s h i ps b et w een i d ea s or t h e s i gn i fi c a n c e of ob j ec t s t h r ou gh t h e pla c i n g of i ma ges . T he re ading p ath of continuous p rint in W e s te rn culture s , runs f rom le f t to rig ht and f rom top to bottom. I n a d i s pla yed t ext , w h er e i ma ges , b loc k s of t ext , t ypefa c e a n d c olou r a r e pla c ed a c r os s a d ou b le pa ge s pr ea d , for exa mple, t h e r ea d i n g pa t h i s n ot s o c lea r . Th e r ea d er h a s a c h oi c e of pa t h w a ys , a lt h ou gh t h er e a r e s ome c on ven t i on s i n Wes t ern t ext or ga n i z a t i on w h i c h c a n d i r ec t t h e r ea d erÕ s eye. Wh en r ea d i n g a mu lt i mod a l d ou b le pa ge s pr ea d t h e eye ma y fa ll a n yw h er e on t h e pa ge, on a s t r on g c en t r a l i ma ge or on a n ot h er s i gn i fi c a n t pa rt of t h e t ext , b u t fr om t h er e, t h e r ea d i n g pa t h w a y i s les s pr ed i c t a b le. Th e eye w i ll r oa m r a d i a lly a r ou n d t h e pa ge, foc u s i n g on a s pec t s of t h e t ext / i ma ges w h i c h s eem i mport a n t a t t h e t i me. Th e ga z e ma y b e d i r ec t ed b y d evi c es s u c h a s a rr ow s or s t r on g vec t or s i n the im a g es , l ea ding the ey e ra ther l ike s ig np o s ting . A c o m ic b o o k ty p e l a y o ut m ig ht direc t the rea derÕs ey e thro ug h s eq uenc es o f f ra m es , o r a s c reen p a g e m a y direc t the ey e thro ug h exp ec ted l a y o ut a nd m enu p l a c ing . F i g u res 5 an d 6 Read i n g an d mak i n g comi c stri ps from ItÕs s erio us l y c o m ic by S arah Abraham an d Rebecca K en n ed y 5 Mult i m od a li ty I n t a c k li n g s c r een � b a s ed t ext s , exper i en c ed r ea d er s t a ke eq u a lly c omplex pa t h w a ys a s t h ey s eek i n forma t i on or c ommu n i c a t e w i t h ot h er s . Sea r c h i n g t h e w eb of t en mea n s z i gz a ggi n g b a c k a n d fort h b et w een s c r een s , ma k i n g s en s e of t h e pr offer ed i n forma t i on t h r ou gh a k i n d of n et w or k of i d ea s . Th e r ea d er h a s t o ma ke d ec i s i on s a b ou t w h a t K r es s t erms ‘ c r i t er i a of r eleva n c eÕ ( K r es s , 2\b\b3) . C ompu t er ga mes t en d t o i n volve s eq u en t i a l a n d r ec u rr en t pa t h w a ys , w h i ls t c ommu n i c a t i n g w i t h ot h er s t h r ou gh w eb s i t es c omb i n es n et w or k i n g moves w i t h t h e c r ea t i on of mea n i n g a s pa rt of a pr oc es s of w r i t i n g a n d i ma ge d u mpi n g. Exper i en c ed r ea d er s ( a d u lt s a n d c h i ld r en ) a r e a lr ea d y c a pa b le of follow i n g va r i ed r ea d i n g pa t h w a ys a n d s o h a ve fu n d s of k n ow led ge w h i c h c ou ld i n form w a ys of t a lk i n g w i t h c ollea gu es a n d t h e lea rn er s t h ems elves a b ou t r ou t es t h r ou gh mu lt i mod a l t ext s . Talk ing ab out multimodality Bei n g a w a r e of t h e d i ffer en t pos s i b i li t i es of mea n i n g offer ed b y mu lt i mod a l t ext s mea n s expli c i t ly d i sc u ssi n g h ow t ext s w or k t o expr ess i d ea s. For exa m ple, i t i s w ort h m a k i n g expli c i t h ow w or d s, i m a ges, d es i gn a n d la you t w or k t oget h er – or s epa r a t ely � t o ma ke a t ext w h i c h gen u i n ely c ommu n i c a t es w i t h t h e r ea d er . Af t er a pr oj ec t w i t h 9 a n d 1\b yea r old b i li n gu a l c h i ld r en w h i c h u s ed mu lt i mod a l t ext s , i n c lu d i n g d r a ma , t o d evelop r ea d i n g for i n fer en c e, Ja n e Bed n a ll a n d Lea n n e C r a n s t on a s ked t h e c h i ld r en t o r eflec t on t h ei r lea rn i n g ( Bed n a ll e t a l., 2\b\b8) . O n e of t h e t ext s t h ey h a d r ea d w a s The R e d T re eb y Sh a u n Ta n a n d t h e c h i ld r en d i s c u s s ed w h a t t h ey t h ou gh t a mu lt i mod a l t ext w a s : Ta s nia : It can be al l types of thi n g s. Y ou can read i n d i fferen t ways, l i k e you can read pi ctu res, scan the pi ctu res i n you r head , you can read from Brai l l e l i k e bl i n d peopl e d o an d you can read from d i fferen t l an g u ag es an d thatÕs what a mu l ti mod al tex t mean s to me. M uha m m a d: W hen you l ook at somethi n g an d you l ook at the si mi l ari ti es an d d i fferen ces between the wor d s an d the pi ctu res. R a s hida : I thi n k i tÕs l ook i n g at both... book s wi th more wor d s than pi ctu res an d more pi ctu res than wor d s an d sayi n g … whi ch on e d o you prefer more? T he Red Tree Õs g ot more pi ctu res than wor d s becau se wi th pi ctu res you can real l y feel the emoti on . Wi th wor d s you can , bu t wi th pi ctu res you can actu al l y see the peopl eÕs faces… I n ev er thou g ht you cou l d real l y read pi ctu res. I thou g ht they were ju st there to mak e i t l ook pretty . J an e [Bed n al l ] ex pl ai n ed i t to u s… S he tau g ht u s that you hav e to pu t the wor d s an d the pi ctu res tog ether . Real l y g o i n to i t… g o i n to the pi ctu re worl d an d real l y an al yse i t, you Õv e g ot to real l y l ook at that person Õs eyes an d thi n k an d thi n k . Figu re s \f a n d 8 Re a d in g a n d a n n ota tin g c om ple x pic tu re b ooks f rom The M os t W onde rful A dv e ntur eb y J a n e Be d n a ll a n d Le a n n e C ra n ston 6 Mult i m od a li ty Wh en a s ked h ow t h ey w ou ld expla i n t o a you n ger r ea d er h ow t o r ea d a c omplex pi c t u r eb ook , Ta n i a Õ s expla n a t i on of t h e pr oc es s s h ow s a n a s t u t e u n d er s t a n d i n g of h ow t h es e t ext s w or k : Ta n i a : Lo o k a t the pi ctu re s ca re f u lly . I f the y ca n Õ t re a d the y ha v e to lo o k a t i t re a lly ca re f u lly a n d lo o k a t the co lo u rs a n d w ha t the pe rso n i s d o i n g – the f a ci a l e x pre ssi o n a n d the b o d y la n g u a g e . Y ou h av e t o l ook at t h e p ic t u r e an d t h e c ol ou r s an d t h e p er s on Õs fac e r eal l y c ar efu l l y t o as k you r s el f a f e w q u e sti o n s: H ow i s t h e per s on feeli n g? Wh a t s ort of s t a t e i s h e or s h e i n ?o r ma y b e a f e w mo re q u e sti o n s. Bu t I thi n k tha t the y sho u ld lo o k a t the pi ctu re re a lly ca re f u lly . . . the co lo u rs a n d the f a ce re pre se n t so me thi n g . T he s e chil dre n s ho w a s o p his ticate d unde rs tanding o f what a m ul tim o dal te x t m ig ht be and ho w wo rds a n d pi c t u r es c a n w or k t oget h er t o ma ke mea n i n g. Home e xpe rie n\fe of multimodal te xts S tudie s o f chil dre nÕs ho m e and s cho o l l ite racy and dig ital e x p e rie nce (Mars h et al ., 2 \b \b 5 ; Cre m in et al ., 2\b15) i n d i c a t e t h a t ma n y c h i ld r en h a ve a c c es s t o ga mes c on s oles , la pt ops a n d PC s , i n t ern et � en a b led tabl e t co m p ute rs o r s m ar tp ho ne s and e x te ns ive e x p e rie nce o f p o p ul ar cul tural te x ts . T he s e are val uabl e as s e ts to build on in the clas s room, but be f ore as s uming what childre n know and can do with multimodal t ext s , i t i s w ort h fi n d i n g ou t t h e fa c t s . O n e pr i ma ry s c h ool of 325 pu pi ls i n Sou t h Ea s t En gla n d d ec i d ed t o s u rvey i t s c h i ld r en t o fi n d ou t a b ou t t h ei r u s e of mu lt i mod a l t ext s , i n c lu d i n g d i gi t a l t ec h n ology, a t h ome. Th e s c h ool w a n t ed t o r evi ew i t s u s e of d i gi t a l t ec h n ology a n d ot h er mu lt i mod a l t ext s i n t h e c l a s s ro o m , s o they a s ked a l l c l a s s es to c o m p l ete a s ur v ey a b o ut their us e o f the f o l l o wing ty p es o f m ul tim o da l texts a t ho m e a nd a t s c ho o l : •c o mi c s •ma ga z i ne s •ne ws p a p e r s •te l e v i s i o n p r o gr a mme s •c o mp ute r ga me s •fi l ms / \bV\bs •i nte r ne t te xts •e � ma i l s •p h o ne te xts •e � b o o ks •b o o ks wi th wo r d s a nd no p i c tur e s •b o o ks wi th p i c tur e s •a ud i o b o o ks W hen the tea c hers rev iewed the res p o ns es , they no ted tha t the c hil drenÕs p o p ul a r c ul tura l ho m e rea ding o f m a g a zines a nd c o m ic s did no t f ea ture hig hl y in the c l a s s ro o m , a l tho ug h the f ig ures f o r rea d i n g n ewspa pers a re a b o u t th e sa me a t h o me a n d a t sc h o o l . Un d ersta n d a b l y, tel evi si o n a n d c o mpu ter game s we re more like ly to be use d at hom e than at s chool, although the figur e for us ing com pute r game s in school was about half that of hom e us e . S cr e e n�bas e d te x ts on the te le v is ion and inte rne t se e me d to be use d e qually re gularly at school and at hom e but e m ails and te x ts , har dly fe atur e d at school. Where children used e� books and a u d io b ooks a t h ome t h ese were mu c h less c ommon in sc h ool, although books, with and without pictur e s w e r e e qually pr e v ale nt at hom e and at s chool. In this school the re se e me d to be re lative ly fe w m aj or diffe r e nce s in m ultim odal te x t us e be tw e e n the boys and the girls. M ore boys than girls r e ad com ics , and m or e gir ls than boy s r e ad m agazine s and newspapers. Tel evision viewing is al most the sam e b e twe e n b o ys and g i rl s wi th the b o ys sl i g htl y hi g he r and where almost all of the boys played c omputer g a mes , the numb er of g ir l s r epor ting pl a y ing g a mes was not far behind. 7 Mult i m od a li ty R evi ew i n g t h e i n forma t i on a s a w h ole, t h e t ea c h er s d ec i d ed on t h e follow i n g a c t i on s : •Wide n the rang e o f re ading m ate rial in cl as s ro o m s to incl ude chil dre nÕs p re f e r re d co m ics , m ag azine s , a u d i o a n d e� b ook s . •Set u p s t a ff t r a i n i n g s es s i on s t o look a t h ow c ompu t er ga mes mi gh t b e u s ed i n t h e c u rr i c u lu m. •Pla n t o u s e ema i ls a n d t ext i n g a s pa rt of En gli s h t ea c h i n g. •Pla n t o u s e fi lm mor e d eli b er a t ely a s t ext s t o b e s t u d i ed , n ot j u s t w a t c h ed . ( F or mor e d et a i ls s ee Bea rn e a n d R eed y, 2\b18: 39\f) De s\frib ing prog re ss in re ading and \fomposing multimodal te xts Na t i on a l c u rr i c u la c ommon ly pr ovi d e fr a mew or k s for d es c r i b i n g pr ogr es s i n r ea d i n g a n d w r i t i n g, a n d s o m e tim e s in s p o k e n l ang uag e . Ho we ve r, the s e do no t us ual l y incl ude re ading / vie wing and co m p o s ing m ul tim o dal te x ts . B e ar ne and B azal g e tte , (2 \b 1 \b ) de ve l o p e d a s e t o f p ro g re s s io n s tate m e nts to de s cribe pr ogr es s i n r ea d i n g/ vi ew i n g mu lt i mod a l t ext s a c r os s t h e d i men s i on s of: •en ga gemen t , u n d er s t a n d i n g a n d r es pon s e t o t ext •i n fer en c e a n d d ed u c t i on •s t ru c t u r e a n d or ga n i s a t i on of t ext s •s t yle a n d c ompos i t i on ( i n c lu d i n g s ou n d ) •pu r pos e, vi ew poi n t a n d effec t of t ext on t h e a u d i en c e •s oc i a l, c u lt u r a l a n d h i s t or i c a l c on t ext . Bea r ne a nd Reedy (2\b18) des c rib e p ro g res s in m ul tim o da l text c o m p o s itio n a s m a rked b y inc rea s ing a b il ity to : •Dec ide o n m o de a nd c o ntent f o r s p ec if ic p urp o s e(s ) a nd a udienc e(s ) c ho o s ing whic h m o de(s ) wil l b es t c o m m unic a te m ea ning f o r s p ec if ic p urp o s es . •St r uc t ur e t ext s b y c o nsc io us a t t ent io n t o desig n a nd l a yo ut , using st r uc t ur a l devic es (p a g es, sec t io ns, f ra m es , p a ra g ra p hs , b l o c ks o f text, s c reens , s o und s eq uenc es , v erb a l , v is ua l a nd s o und c o hes iv e dev ic es ) to o rg a nis e texts . •H a ndl e tec hnic a l a s p ec ts a nd c o nv entio ns o f dif f erent kinds o f m ul tim o da l texts , inc l uding l ine, c o l o ur, p ers p ec tiv e, s o und, c a m era a ng l es , m o v em ent, g es ture, f a c ia l exp res s io n a nd l a ng ua g e. •E xp l a in c ho ic es o f m o des (s ) a nd exp res s iv e dev ic es inc l uding wo rds a nd im p ro v e c o m p o s itio n o r p er f o r m a nc e, res ha p ing , redes ig ning a nd redra f ting f o r p urp o s e a nd rea ders Õ/ v iewers Õ needs . \b tarti n g earl y C hil drenÕs exp erienc e o f m ul tim o da l texts s ta r ts v er y ea rl y. In the E a rl y Y ea rs F o unda tio n Sta g e (E YF S) they o f ten dra w o n a wea l th o f a s s ets in ter m s o f their exp erienc e o f wa tc hing , l is tening , hel p ing a nd l ea r ning ho w div ers e l itera c y res o urc es a re o rg a nis ed a nd a c c es s ed. N o t o nl y a re they exp erienc ed – a nd c ritic a l – rea ders / v iewers o f tel ev is io n a nd f il m , b ut they a re l ikel y to ha v e a c c es s ed iP a ds , m o b il e phone s and playthings which use digital te chnologie s . A ls o, the ir e x pe r ie nce of e nv ir onm e ntal pr int and image , and of be ing in social se ttings , m e ans that the y k now about how diffe r e nt or al, s y m bolic and printe d te xts work to make me aning. The r e ar e conce rns , how e v e r , that s om e childr e n s pe nd too much time at home with scre e ns so that this is some thing that a) ne e ds to be discove re d by discussion with pare nts/care rs; and b) can be me diate d in the se tting in orde r to dire ct childr e n to othe r kinds of multimodal activitie s inv olv ing b ooks, toys, puppets, dressing up cl othes, m usi cal instruments and outdoor discovery. Imag ina tiv e outdoor play is intensely multimodal, inv ol v ing a range of materials and opportunities to shap e i de as. Figure 9 Leaf pai nt i ng 8 Mult i m od a li ty Abo ve; F igu re 10 W o rkin g in t h e m u d kit c h en Righ t ; F igu re 11 D res s in g u p Far righ t ; F igu re 12 C o u n t in g u s in g Makat o n T here are im pl icatio n s f o r practice reg ard in g the w ay s ad u l ts s u ppo r t y o u n g chil d ren in beco m in g l iterate in their u s es o f n ew m ed ia. C hil d ren Õs f u n d s o f k n o w l ed g e abo u t m u l tim o d al tex ts are l arg el y im pl icit, s o chil d ren n eed g u id an ce an d s caf f o l d in g f ro m ad u l ts to s ho w them ho w they can u s e their ex perien ce o f the co m pl ex f o r m s o f m u l tim o d al co m m u n icatio n f o r f u r ther l ear n in g . H o w ev er, there can be ten s io n s abo u t this as s tu d ies (W o l f e an d Fl ew itt, 2\b1\b) s ho w that m an y practitio n ers an d s taf f l ack the co n f id en ce an d ex per tis e to d o this . I n s o m e in s tan ces , ad u l ts m ay appear as n o v ices in me di a t i ng ne w di gi t a l r e sour c e s, whe n t he y ne e d t o b e ski lle d a nd c onfi de nt i n b ui ldi ng up on c hi ldr e nÕ s f u n d s o f k n o w l ed g e T here are al s o pro f es s io n al is s u es abo u t po s itio n in g d ig ital techn o l o g y in E YF S cl as s ro o m s / s ettin g s , f o r ex am pl e, a d es k to p co m pu ter o ften pu s hed to the co r n er o f the ro o m , w hereas o ral s to r y tel l in g / bo o k s harin g is n o r m al l y o f f ered acro s s the s ettin g – o n the carpet, in d o o rs / o u td o o rs , o n tabl e�to ps , under t ab les … t he li s t g o es o n. It i s wo r t h havi ng a lo o k at t he p laci ng o f di g i t al t echno lo g i es i n relat i o n to other resourc es; ra ther tha n siting in in th e c or n er of a r oom, it sh ou l d be c en tr a l l y pl a c ed so th a t it c a n bec ome the hub of soc ia l a c tivity, a l l ow in g a d u l ts a n d c h il d r en to sh a r e th e ex per ien c e of u sin g the tec hnol ogy – just a s they woul d when sh a r in g a book. Observing and re\fording young \f hi l d r enÕs mul ti mo d al \f o mmuni \f ati o n Before c hil dren rea d a nd write orthodox tex ts, th ey a r e n ever th el ess en ga ged in a r a n ge of d iffer en t way s of communicating and re pre se nting the ir id e a s. I n r e c or d in g c hil d r e n Õs a bil itie s in c ommu n ic a tion , it is worth ta king time to observe how ga ze, movemen t, gestu r e a n d position in g c on tr ibu te to you n g childrenÕs multimodal expression and dialogu e. I t i s u sefu l to ob ser ve ch i ld ren Õ s d i alogu e i n play an d particularly to note their developing talk, th ei r early w ri ti n g ex peri m en ts an d th ei r stor ytelli n g as th ey ‘readÕ a book. It is less usual to note how th ei r gestu re, gaze an d m ovem en t also con tri b u te to th ei r development as communicators. For exam ple, w atch i n g a ch i ld pori n g over a b ook , d i recti n g h er/ h i s gaze to different images, or observing ch i ld ren sh ari n g a b ook , poi n ti n g to elem en ts of th e page or Fig u re s 9 t o 1 2 f r o m St. P ete r's R C P rim ary Sc h o ol, S itti n gb o u rn e, K en t, w ith t h a n ks t o C ath erin e V ed a m uttu 9 M ult i m od a li ty w or k i n g t oget h er t o u n d er s t a n d t ext s , c a n r evea l a gr ea t d ea l a b ou t w h a t t h ey u n d er s t a n d a b ou t c h a r a c t er , fa c t u a l i n forma t i on or t h e w a y t ext s w or k . Si mi la r ly, w a t c h i n g a c h i ld d r a w i n g a n d w r i t i n g, pa rt i c u la r ly t h e pa u s es a n d gla n c es a w a y fr om t h e pa ge i n d i c a t e very c lea r ly t h e pr oc es s of t h ou gh t a s t h e w r i t i n g/ d r a w i n g i s c ompos ed . I n i ma gi n a t i ve pla y, a n d pa rt i c u la r ly d r es s � u p pla y, pos t u r e a n d u s e o f the g ar m e nts – s wirl ing a cl o ak, ho l ding a s kir t, we aring a hat – can re ve al no tio ns o f characte ris atio n a n d s t ory t h eme. Le ft; Fi g u re 1\b Wo rk i n g o n w o rd s f ro m We stla n d s f i rst scho o l, D ro i tw i ch, w i th tha n k s to Se re n Fre e sto n e Be lo w ; Fi g u re 14 C o mpo si n g i d e a s f ro m St. Pe te r's R C Pri ma ry Scho o l, Si tti n g b o u r n e , K e n t, w i th tha n k s to C a the ri n e V e d a mu ttu I mpl i \fati o n s f o r pra\fti \fe If y o ung p eo p l eÕs text exp erienc e is to b e g enuinel y rec o g nis ed in the c l a s s ro o m then it is im p o r ta nt to dis c o v er jus t wha t they kno w a b o ut the texts they enc o unter ins ide a nd o uts ide s c ho o l a nd the m a ny wa y s they m ig ht rep res ent their idea s . L itera c y tea c hing no w m ea ns tea c hing a b o ut m ul tim o da l ity, inv o l v ing : •und e r s ta nd i ng ho w texts a nd m o des wo rk •d e mo ns tr a ti ngthis unders ta nding in the c l a s s ro o m •encour a ging le arne r s to us e m ultim odal r e pr e s e ntation to s hape and com m unicate the ir ide as •helping them to d evelop a re pe rtoire of appr oache s and the n be s e le ctiv e in m atching m ode w ith purpose and in making appropriate choice s •provid ing opportunities for working w ith a r ange of m ate r ials . Howe ve r, the task is not only to e ngage in dialogue s w hich w ill he lp le arne r s r e cognis e the diffe r e nt re pre se ntational de mands made by diffe r e nt te x ts . It is e qually im portant to de v e lop a com m unity of professional experience ab out mul timodal i ty, to o b se r ve chi l dre n as the y wo rk wi th di ffe re nt m ate ri al s, mode s and me chanisms, to le arn how to de s cr ibe and r e cor d pr ogr e s s in m ultim odal r e pr e s e ntation and communication, and to consider how y oung r ea der s a nd v iew er s c a n b e hel ped to r ea d c r itic a l l y beyond the images as well as reading betw een the l ines of v er b a l text b oth on s c r een a nd of f . 1\b M ult i m od a li ty Re f e re n\fe s A b r a h a m, S. a n d Ken n ed y, R. ( 2\b1\b) ‘ I t Õ s s er i ou s ly c omi c Õ : R ea d i n g a n d ma k i n g c omi c s t r i ps , i n E. Bea rn e a n d C . Ba z a lget t e ( ed s . ) Be y o n d Wo rd s. Lei c es t er : Un i t ed K i n gd om Li t er a c y A s s oc i a t i on . Bea rn e, E. a n d Ba z a lget t e, C . ( ed s . ) ( 2\b1\b) Be y o n d Wo rd s: D e v e lo pi n g chi ld re n Õ s re spo n se to mu lti mo d a l te x ts. Lei c es t er : Un i t ed K i n gd om Li t er a c y A s s oc i a t i on . Bea rn e, E. a n d R eed y, D . ( 2\b18) Te a chi ng Pri ma r y E ng li sh: Sub j e ct Kno w le dg e a nd Cla ssro o m Pra cti ce. A b i n gd on , O xon . : R ou t led ge. Bed n a ll, J. , C r a n s t on , L. w i t h Bea rn e, E. ( 2\b\b8) Th e M os t Won d erfu l A d ven t u r e… goi n g b eyon d t h e li t er a l, i n E n g li sh 4� 11 ( 32) pp. 19� 26. Su k h vi n d er Bh od a y a n d Ja n e Bed n a ll ( 2\b17) Th e Ts u n a mi pr oj ec t : w r i t i n g n a rr a t i ve poet ry, i n E n g li sh 4� 11 ( 59) Spr i n g, 2\b17 pp. 13� 16. Cre m in, T ., Mo ttram , M., Co l l ins , F., P o we l l , S . and Dr ur y , R . (2 \b 1 5 ) Res ear c h in g Lit er ac y Liv es : B u il din g co mmu n i ti e s b e tw e e n ho me a n d scho o l. A b i n gd on , O xon . : R ou t led ge. K r es s , G. ( 2\b\b3) L i te ra cy i n the Ne w Me d i a A g e. Lon d on : R ou t led ge. M a r s h , J. , Br ook s , G. , H u gh es , J. , R i t c h i e, L. , R ob ert s , S. a n d W r i gh t , K . ( 2\b\b5) D i g i ta l Be g i n n i n g s: Y ou n g P eop l eÕs U s e of Pop u l ar C u l t u r e, M edia an d N ew T ec h n ol ogies. Unive rs ity o f S he f f ie l d: S he f f ie l d. A v ai l abl e o n : h ttps: //issu u . com /esm eef airbairn /docs/dig ital beg in n in g s�report ( A ccessed 23r d Ju n e. 2\b19) W olf e , S . a nd F le w i t t , R. ( 2 \b 1 \b ) Ne w t e c hnolog i e s, ne w mult i moda l li t e r a c y p r a c t i c e s a nd young c hi ldr e nÕ s m eta c o g nitiv e dev el o p m ent, C ambri d g e J ou r n al of Ed u cati on , \f\b(\f), p p .387�399. A\fkn o wl edg emen ts T ha nks to c hil dren a nd tea c hers a t: C l if to n Prim a r y Sc ho o l , Ba l s a l l H ea th, Bir m ing ha m E l m hurs t Prim a r y Sc ho o l , N ewha m , L o ndo n St. P eter's RC Prim a r y Sc ho o l , Sitting b o ur ne, K ent Wes tl a nds Firs t Sc ho o l , Dro itwic h, Wo rc es ters hire \fe d ed icat e t his paper t o G u n t her K res s (1 9 \b 0 � 2 0 1 9 ) w hos e w ork has in s pired u s in ou r u n d ers t an d in g of m u l t im od al it y. 11 M ult i m od a li ty