Stony Brook’s Reality Deck Offers High Schoolers a Glimpse of the Future

Arie Kaufman describes the Reality Deck located in the Center of Excellence in Wireless and Information Technology (CEWIT) at Stony Brook as a dream he carried for a decade before finally getting the funding to build it. Ten years after its completion in 2012, the immersive display — the largest of its kind in the world — serves as a valuable virtual reality training ground for high school students and a feeder for Stony Brook’s computer science undergraduate program.

CEWIT’s Reality Deck is a 30-foot-by-40-foot room covered with 416 27-inch screens, each with 4 million pixels, that immerses scientists, researchers and students inside a high-definition, 1.6-billion-pixel world.

“The vision has been to give everyone from the age of 5 to 105 on Long Island an opportunity to learn computer programming,” said Kaufman, chief scientist of CEWIT and a distinguished professor in Stony Brook University’s Department of Computer Science in the College of Engineering and Applied Sciences.

Kaufman and the Department of Computer Science have collaborated with kidOYO®, an education-focused nonprofit that builds mentor-led communities, prioritizing computational literacy for production-grade learning standards. The organization supports students in grades K-12; it connects them with opportunities in computer science and helps them develop entrepreneurial imagination and skills while the university’s students serve as mentors.

Kaufman was especially interested in working with kidOYO because the organization has close to a 50-50 male-female ratio.

“We need to make our program more attractive to female students,” said Kaufman. “It’s about building critical mass; we don’t want to have a single female student sitting in the back of the classroom. This was an opportunity to build a relationship for gender equality and have our own students serving as mentors.”

Earlier this year, kidOYO joined as an industrial member of the National Science Foundation’s Industry University Cooperative Research Center in the Center for Visual and Decision Informatics, which is directed by Kaufman. This new collaboration enables local high school students with an interest in coding and research to take advantage of Stony Brook’s facilities, while Kaufman’s PhD students serve as their mentors.

Through annual summer internships, Kaufman and kidOYO hope to spread the virtues of coding and computer programming as well as computer science research to high school students across Long Island.

“We want to get our program in front of more high school students who are well-trained in coding,” Kaufman said. “What they learned here was how to conduct computer science research. We want them to go back as ambassadors to their peers and tell them about their experience at Stony Brook.”

This summer, Kaufman and his PhD students hosted two high school juniors with interests in immersive virtual reality and augmented reality. Three PhD mentors worked closely with them.

“When they saw the Reality Deck, they said, ‘we’re not going anywhere, we’re staying right here,’” said Kaufman. “The time that we spent with them was a lot of fun. The complexity of what they did is exceptional.”

The reality deck displays the city of Dubai in the United Arab Emirates.

“Working with these students was awe-inspiring,” said Saeed Boor Boor, a PhD candidate in Computer Science. “The creativity, spectacular problem-solving skills, and undeterred determination in these two students, who are the protagonists of our global future, were nothing less than phenomenal. It was a delight to work and learn alongside them.”

The students created an interactive paint program and an augmented reality-based collaboration interface that they demonstrated at the Reality Deck on August 24.

“Visual computing is attractive to high school kids because the visual feedback they see is like the games they played growing up,” he said. “We don’t teach games specifically, but the future of all of this is going to be augmented or virtual reality, and you will be able to play games in those settings.”

One student who attended the program is getting a pilot’s license, and Kaufman asked him why. “He said he played so many video games about navigating airplanes virtually that he was familiar with how to approach it even before he went to take the actual lessons.”

The next phase of Kaufman’s research using the Reality Deck will focus not only on visual computing and big data, but also on visual analytics and machine learning — using the visual images and graphics to make real-life decisions as well as using the Reality Deck graphics processing units as accelerators for machine learning. Kaufman and his team will collaborate with kidOYO to develop high school curricula in visual computing and machine learning.

“How do you use graphics to show you what is an outlier and what should be the trend?” Kaufman said. “We do a lot of extrapolations. There are a lot of things that graphics like these can help with. The visual channel is the widest one. Why not exploit it?”

Kaufman said he expects interest in the summer internship to increase, and he hopes to host twice as many students next year.

“If we double it, I will be very happy,” he said. “We’re also planning an open house for this year’s students to demo their project for school administrators, teachers and fellow students. People will see the facility and the demo, and we hope that it will generate much interest. This has been a long-awaited dream for me. Having students of this caliber in this facility is very gratifying.”

– Robert Emproto