Got Interactive (Multimedia) Textbooks Inside Your iPad or Tablet? Lots of Links!

Reposted from the Interactive Multimedia Technology Blog:

If you live in the US, and haven't yet heard, the U.S. government is encouraging K-12 schools to adopt interactive digital textbooks within the next five years.  Are we ready?  I don't think so.  There is a battle of the tablets going on as I type this post.  

Traditional, "old-school" textbook publishers appear to be key players in this game, but I am not sure if they have the know-how to create interactive content that is fresh, "touchable", engaging, and meaningful to young people.  A boring textbook, tweaked tweaked for a bit of interactivity and multimedia content will not suffice.

Feel free to take your time and visit the links I've shared so far. I'm still reflecting on what I've learned so far and will share more of my thoughts in future related posts.

Leaders Discuss Transition to Digital Textbooks
The LEAD commission 

Nation's Digital Learning Report Card (Interactive map)
Digital Learning Now!

US FCC: Digital Textbook Playbook (website)

Digital Textbook Playbook (pdf)
The Digital Textbook Collaborative, 2/1/12


Slide Presentation: Charting our Transition to "Interactive Digital Textbooks"  (pdf)
FCC LEAD Commission (Leading Education by Advancing Digital)
Video: Remarks by (FCC) Chairman Genachowski and Digital Education Leaders on National Adoption of Digital Textbooks

Key players: 
Dan Caton, McGraw-Hill
Will Ethridge, Pearson
Jose Ferreira, Knewton
Bill Goodwyn, Discovery Education
Joel Klein, News Corp
Matt MacInnis, Inkling
Osman Rashid, Kno
Dan Rosenweig, Chegg 
Linda Zecher, Houghton Mifflin 
Harcourt


Arne Duncan Calls for Textbooks To Become Obsolete in Favor of Digital  
Josh Lederman, Huff Post Education 10/2/12


Kno Interactive Textbooks App Now Available for Google Nexus Tablet and Google Play 
Business Wire, 10/29/12  Kno

Teaching with tablets: Will our children be using electronic textbooks to learn?  
Jamie Carter, 10/19/12

In digital textbook transition, device availability is just the beginning
Ki Mae Heusser, Gigaom, 10/22/12

Amazon challenges Apple in education with Whispercast
Ki Mae Heussner, Gigaom, 10/17/12

To empower students, let's bring interactive learning tools into the classroom  
Troy Williams (President of Macmillan New Ventures) Venture Beat, 10/30/12

Press Release: McGraw-Hill Ryerson Launches New High School iBooks Textbooks Digital Book Wire, 10/25/12

South Korea Classrooms to go fully Digital by 2015
Ben Gruber, Reuters, 10/5/12


South Africa: Launch of OS-Agnostic Digital Textbook Publishing Platform 
All Africa, 10/23/12

Whitepaper: From Paper to Pixel: Digital Textbooks and Florida Schools (pdf) 
Marcia Mardis, Nancy Everhart, Daniella Smith, Janice Newsum, Sheila Baker, Florida State University PALM Center, 2010

Whitepaper:  The Digital World of Young Children: Imact on Emergent Literacy (pdf))  Jay Blanchard, Terry More, Arizona State University, 2010 (Includes demographics about digital media survey data in selected developing and least developed nations.)

Apple Pushes Interactive Textbooks on iPads  
NPR Staff and Wires, 1/19/12
"Forrester Research said e-books accounted for only 2.8 percent of the $8 billion U.S. textbook market in 2010."

Engage: Apples New Tools for Interactive Books on iPad  
Tim Carmody, Wired, 1/19/12

Inkling Reinvents Textbooks as Interactive, Multimedia Learning Tools on the Tablet  David Weir, 7x7 SF, 7/1/11
Inkling

iBooks Author (free app)  -Apple

What's New in iBooks Author 2.0 -Apple

Houghton Mifflin Harcourt Press Release: Student Math Scores Jump 20 Percent with HMH Algebra Curriculum for Apple iPAd; App Transforms Classroom Education1/20/12HMH Fuse Pilot Program (Website)

SOMEWHAT RELATED
Digital Book World (Website)
MediaShift: Your Guide to the Digital Media Revolution (PBS website)
The Transition to Digital Journalism  Paul Grabowicz, Knight Digital Media Center, Berkeley, 10/26/12
How to make BYOD (Bring Your Own Device) work for your schools Laura Dvaney, eSchool News, 10/29/12
BLOG: Mind/Shift: How we will learn (KQED)
iBooks Author

iPad3 and iPad Mini: Hands-on Side-by Side Comparison Video, by Eric Sailers (quick post)

Here is a good side-by-side "hands-on" comparison of the new iPad3 and the new iPad Mini by Eric Sailers:

Eric Sailers is a speech and language pathologist who has co-created apps for iOS devices since 2009. His website has a wealth of information iOS devices and apps for education, especially for children with special needs.  

Quick Link: 68 interesting ways to use an iPad in the classroom - Josh Fassbind

68 Interesting Ways to use an iPad in the Classroom

Quick Link and Video: InfoVis of Social Interactions of Children at School

Stehlé J, Voirin N, Barrat A, Cattuto C, Isella L, et al. 2011 High-Resolution Measurements of Face-to-Face Contact Patterns in a Primary School. PLoS ONE 6(8): e23176. doi:10.1371/journal.pone.0023176

The article that explains it all:

Visualized: A School Day as Data

Brandon Keim, Wired Science, 8/19/11 

The video: Contact patterns between the students. (Barrat et al./PLoS One)

Thanks to George Siemens for sharing this video!

EyeTube for YouTube! Eye-gaze interaction software, free and downloadable from GazeGroup

Gaze interaction systems provide access to computers and the rich content now available on the web for many people with disabilities.  Unfortunately, commercial gaze tracking systems are very expensive and at times, difficult to calibrate.  There is hope!



Following up on my recent post, "Open-source Eye-tracking: The ITU Gaze Tracker 2.0 Beta", I thought I'd share the GazeGroup's EyeTube for YouTube interface.  What is great about EyeTube for YouTube is that it provides two different interfaces. The simplified version looks good for younger children or people with cognitive disorders, and is icon-based.  The second version is appropriate for people who can navigate through more complex visual representations of content. EyeTube requires a Windows-based system and .Net 3.5 at this time. It can be downloaded from the GazeGroup website.  If you plan to download the application, you must also make sure you have a YouTube account. To get the application up and running, you'll need to change the settings (EyeTubeSettings.xml) to match your account.   (If you don't know much about changing settings or xml, ask someone you know who works in IT.)
Below is the icon-based version of the eye-gaze interface for YouTube:


Feature-rich version of the EyeTube interface for YouTube:


From the GazeGroup site:
"The EyeTube prototype offers a feature rich eye controlled interface for the popular YouTube service. Instead of emulating a mouse pointer and interacting with a web browser the EyeTube interface is especially designed to be driven by gaze input. It offers a wide range of features such as keyword searching, popular video feeds, favorites and social aspects such as subscriptions, friends and commenting on videos.The highly optimized interfaces allows for a streamline interaction which is aleviated from the Midas Touch problem. In most previous gaze interfaces selection is made by a dwell time activator, e.g fixat a button for a specific amount of time and it will execute the function. In the EyeTube interface a fixation on a U.I element will highlight it and a second fixation on the activation button is required to execute the function. This removes the stress of having to constantly move the eyes to avoid unintentional activation."

"The EyeTube also exists in another simplified incarnation developed for users whom are distracted by a larger number of options. It supports basic features such as browsing categories, optional keyword searching and favorites."

RELATED
The GazeGroup
(The individuals mentioned below may be currently working elsewhere, but involved in the gaze research in some way.)

• John Paulin Hansen, Head of group (Ph.D. in Psychology)
• Dan Witzner Hansen, Associate professor
• Javier San Agustin, PhD student
• Sune Alstrup Johansen, PhD student
• Henrik Hegner Tomra Skovsgaard, PhD student
• Martin Tall, PhD student (Now working at Duke University, North Carolina)

GazeGroup Research Areas

Development of Gaze Tracking Technology

Application of Eye Tracking for User Experience studies and HCI

Gaze based Interaction

COGAIN (Communication by Gaze Interaction)

ACM CHI Conference Articles
San Agustin, J., Skovsgaard, H., Hansen, J. P., and Hansen, D. W. 2009. Low-cost gaze interaction: ready to deliver the promises. In Proceedings of the 27th international Conference Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI EA '09. ACM, New York, NY, 4453-4458. DOI= http://doi.acm.org/10.1145/1520340.1520682
San Agustin, J., Hansen, J. P., Hansen, D. W., and Skovsgaard, H. 2009. Low-cost gaze pointing and EMG clicking. In Proceedings of the 27th international Conference Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI EA '09. ACM, New York, NY, 3247-3252. DOI= http://doi.acm.org/10.1145/1520340.1520466 
Tall, M., Alapetite, A., San Agustin, J., Skovsgaard, H. H., Hansen, J. P., Hansen, D. W., and Møllenbach, E. 2009. Gaze-controlled driving. InProceedings of the 27th international Conference Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI EA '09. ACM, New York, NY, 4387-4392. DOI= http://doi.acm.org/10.1145/1520340.1520671

UPDATE

Eye-controlled games and leisure applications from the COGAIN wiki: http://www.cogain.org/wiki/Leisure_Applications

• EyeArt - EyeArt eye-drawing program, developed by Andre Meyer and Markus Dittmar, Technical University of Dresden, Applied Cognitive Research Unit, Germany.
• GazeTrain - Gaze-controlled action oriented puzzle game, developed by Lasse Farnung Laursen, Technical University of Denmark
• Puzzle - Simple puzzle game that can be played with eye movements, developed by Vytautas Vysniauskas, Siauliai University, Lithuania
• Road to Santiago - Gaze-controlled adventure game (full game), developed by Javier Hernandez Sanchiz, Universidad Publica de Navarra, Spain
• Snap Clutch - An application that uses eye gaze data to generate key and mouse events for playing games such as World of Warcraft and Second Life.
• ASE: Accessible Surfing Extension for Firefox - Follow this link to access ASE, an Accessible Surfing Extension for Firefox, developed by Emiliano Castellina and Fulvio Corno at Politecnico di Torino. (Note that this is a beta version.)
• Eye Gaze Music (SAW Selection Sets) - Point and Play – eye gaze (direct pointing) musical activities, developed by DART. Please note that SAW (Special Access to Windows) framework application is needed to play these 15 music selection sets. SAW is available for free athttp://www.oatsoft.org/Software/SpecialAccessToWindows
• EyeTube - Gaze interaction for YouTube - Follow this link to get more information and download EyeTube at ITU GazeGroup's web pages
• Eye3D and other head eye mouse software - Eye3D for education, and a collection of links to free software that works with head or eye mouse. Includes links to downloads and original sites.
• Gaze-controlled Breakout - Follow this link to access a modified version of the LBreakout2 game which can be operated by an SMI eye tracker, developed by Michael Dorr et al. at University of Luebeck
• Oleg Spakov's Freeware games for MyTobii - Follow this link to access MyTobii compatible games developed by Oleg Spakov, University of Tampere, Finland
• Free ITU Gaze Tracker and applications - Download a webcam based open-source gaze tracker and several applications that work with it, developed at IT University of Copenhagen
• GameBase - Check out the Eye-Gaze Games category at the SpecialEffect GameBase!
• More information about Gaze-Controlled Games - Follow this link to see a list of online information resources on using gaze for the control of games and other leisure applications

Cross-posted on the Interactive Multimedia Technology blog.

Establishing Joint Attention with Students with Autism Spectrum Disorders using the HP TouchSmart PC and NUI Suite Snowflake Software

I've been using my HP TouchSmart PC at work with students with disabilities. I'm experimenting with the NUI Suite SnowFlake on my TouchSmart.

The software utilizes a plug-in from NextWindow that allows for more than one touch on the screen at a time. I found that interacting with the Particles application delighted students with severe autism.

I found that many of the activities included in the Snowlake software provided opportunities for the students to establish joint attention. I also noticed an increase in the number of vocalizations and/or verbalizations among the students. Of course, this was NOT a scientific study.

NUI Suite Snowflake Multi-Touch Applications (Demonstrated on a touch-table, but the effect is similar on the HP TouchSmart.)

Note: The Particle application starts at 3:00 - the demo is set to show how it could be used as a social activity in a bar, but it is clear to see the possibilities for use with students.

Update: Here is the SnowFlake application running on a TouchSmart:


More about the HP TouchSmart PC in education:
Interacting and Communicating with HP TouchSmart Notes: Photos, Video, Audio, and More


Joint Attention:

Definition of Joint Attention from UConn:
"Joint Attention is the process of sharing one’s experience of observing an object or event, by following gaze or pointing gestures. It is critical for social development, language acquisition, cognitive development…"

Establishing joint attention is an important step in the development of social interaction skills among young people who have autism spectrum disorders.

Joint Attention Study Has Implication for Understanding Autism Science Daily, 9/29/07
Asperger-Advice: Joint Attention
Autism Games: Joint Attention and Reciprocity
Why is joint attention a pivotal skill in autism?
Tony Charman

Philos Trans R Soc Lond B Biol Sci. 2003 February 28; 358(1430): 315–324. doi: 10.1098/rstb.2002.1199.

Gesturetek Health: "Interactive Technology & Virtual Reality Systems for Therapy & Rehabilitation"

(Pictures from the GestureTek website.)

Gesturetek's Health division has created a variety of interactive applications designed "... for the disability, hospital, mental health and education sectors. Our touch-free, motion-activated solutions for virtual reality physical therapy, multisensory stimulation and immersive play let patients (regardless of age, ability or condition) have fun in a hygienic environment and experience marked improvement in their physical and cognitive abilities."

Here are a few highlights:




Overview of some demos from GestureTek:


References:
GestureTek Health and FlagHouse Announce Partnership to Bring Sensory Solutions to the Physically and Cognitively Challenged

GestureTek Videoclips

GestureTek Company Fact Sheet (pdf)

Note: Dean LaCoe, of GestureTek, will be presenting at the Interactive Displays 2009 Conference in April.

Interacting and Communicating with HP TouchSmart Notes: Photo, Video, Audio, and More

via Interactive Multimedia Technology

The HP TouchSmart Notes application can be used in a variety of interesting ways. I have an HP TouchSmart PC, and I like how easy it is to slide the notes about the screen using this multi-modal application. It allows the user to take capture pictures, video, or audio to create notes, as well as traditional "stickies" that allow for typing or finger-writing input. You can even draw or write over photos. It is an application that has many accessibility features. It also supports many of the principles of Universal Design for Learning (UDL).

The various notes can be put together in a variety of ways, and allow for video or photo storytelling and interesting ways of leaving messages- even a song or two.

I did some hunting and found the more information about some non-verbal students with autism who are learning verbal skills through this technology at Hope Technology School in Palo Alto, California.

Video of student using the HP TouchSmart Notes application at Hope Technology School:






Here are a few "How Two" videos that demonstrate the TouchSmart Notes features and interactions:

HP TouchSmart Voice Notes


TouchSmart Text Notes


The pictures below are from the HP TouchSmart Notes Application Review , by Peter Redmer 11/14/08



Text/Drawing Note Creation Panel


Icons for selecting input method for notes



Photo notes input panel:



You can draw on your photo-notes:

If you are looking for information about brain-computer interfaces, follow the link to my post about Emotive Systems neural interface on the Technology-Supported Human-World Interaction blog. It looks like it holds promise for cognitive rehabilitation applications and games.

Emotiv System's Neural Game Controller Headset: Human-Computer Interface of the Future?



Also see:
Game Interaction via Thoughts and Facial Expressions: EPOC - Emotiv Systems Neural Interface

Virtual Tutors: Link to article about Justine Cassell's work with children with autism spectrum disorders

Justine Cassell, director of the Center for Technology and Social Behavior of Northwestern University, is the developer of the "Embodied Conversational Agent", a virtual human designed to interact children with autism. Her work was highlighted in a recent article in the technology & science section of the MSNBC website:

The ArticuLab, Northwestern Univ
"A “virtual child” is a cartoon about the size of an 8-year-old with whom kids can learn
and play on the floor with toys via a plasma screen projection."

"Virtual characters and digital tutors are helping children and adults develop advanced social and language skills that can be tough to learn via conventional approaches"

Related Links:
Eve: An Understanding Virtual Teacher