Testers, Storytellers, and Cultural AI
ConTEST brought together QA and Testing leaders to New York City to engage in emergent trends and practices in software testing and quality.
How can testers prepare for the future of AI, testing, and personalization? That’s what we workshopped on Friday at ConTEST NYC led by Test Master Academy’s Anna Royzman in New York City.
As the Founder and Storyteller in Chief of IVOW AI, I was invited to showcase our research in cultural IQ as well as our global dataset challenge — An Algorithm for Stories on Women. The focus of our workshop was how to prepare for the emergence of cultural intelligence in AI.
As an automation architect for Discovery (USA), Brian Saylor oversees the planning and strategies for testing and data integrity automation for websites like FoodNetwork.com and TravelChannel.com. His group came up with these two ideas:
1) Build standardized datasets which include data from multiple cultures that can be shared with companies. In this way we will have easy access to bias-free data for training machine learning algorithms.
2) Build a tool that can test datasets for bias. When custom datasets are created for training, they can be tested to determine the types of cultural or other biases that may exist in the data.
Yes! For the field of AI and personalization to be effective and culturally relevant, we must create comprehensive datasets to nurture cultural intelligence in machines and even in social robots like Sophia of Hanson Robotics, who has traveled to over 30 countries. Each time Sophia travels, she meets people of all different backgrounds who talk to her. Collecting culturally prominent datasets in an efficient and scalable manner today is paramount to future commercial success of any AI solutions and products and inclusive AI.
Other testing leaders present included:
- Angie Jones Automation Architect, Senior Developer Advocate at Applitools (USA)
- Shesh Patel The New York Times Engineering Manager
- Tanya Kravtsov Director of QA at Audible
- Jason Huggins Founder of Tapster Robotics (USA)
Adam Mardula is a Test Engineer at Business Insider in New York City. He workshopped with a team that suggested creating a sandbox / testing environment where testers can interact with the current AI technologies that are out there — like robots.
“In this case,” Mardula says, “we can potentially just have one component of the Sophia project available in the sandbox: Emotion tone interpretation.”
Mardula said testers could have the opportunity to speak to the robot with frustration, urgency, excitement, fear, happiness, and so on, and the sandbox should theoretically be able to provide the appropriate feedback about what was spoken. “In this way we can have an environment in which tests can be run. Enter a speech snippet spoken with some emotion, and the correct feedback about the emotional state of the speaker should be returned. The same results should be returned from speakers of different backgrounds, cultures, and even languages,” Mardula said.
Priyanka Halder is Sr. Manager Quality engineering, GoodRx. She came up with an idea to launch an AI based hackathon in Los Angeles. It will be a great opportunity to train the QA community on ML/AI modeling through various hands on session and give them the opportunity to apply the learning on few LA based startup products very next day. “I am super excited to teach the QA community how to test non deterministically and be confident about our future,” Priyanka said.
It was exciting to see the enthusiasm in the room as teams from all over the world imagined this future. At IVOW AI, we have joined Test Master Academy as part of our global dataset challenge to crowdsource and make available open data on stories of women in history, culture, science and technology. This collection is necessary to provide critical historical context for the preservation of culture with an emphasis on the role of women.
This is the first of ten global AI and Storytelling challenges between 2019–2029 that will introduce stories to AI and advance cultural intelligence in AI systems. Future datasets will focus on (a) stories of healthcare and heritage; (b) stories of indigenous people and nature; © stories of people on the Autism Spectrum; (d) and stories of refugees.
“We are excited to join forces with IVOW AI to make the first steps in bringing inclusiveness and quality to the future of our technological society,” says Anna Royzman of Test Master Academy. “This collaborative project will offer multiple opportunities for the global software testing community to learn and be involved in a hands-on project making a positive impact on humanity.”
The goal of the first dataset challenge — An Algorithm for Stories on Women — is for participants to develop an algorithm that can generate a character profile when provided the name of a prominent female in history, science, technology, or culture including folklore and myth. This algorithm is intended to scrape information from various sources off the internet and generate a character profile which includes a caption that is fewer than 100 words, and responses to metadata tags.
Beginning December 18 and through May 2020, IVOW AI and Test Master Academy together with Senior Software Engineer Aprajita Mathur plan to hold engaging webinars to provide expertise and information on testing and AI. Workshop participants gave their ideas on what they would like these trainings to cover:
The team leading our global dataset challenge for stories on women includes: Davar Ardalan, Aprajita Mathur, Professor Ioannis A. Kakadiaris, Amir Banifatemi, Kashyap Coimbatore Murali, Anna Royzman, Sharada Mohanty, Camille Eddy, Nishan Chelvachandran, Mariana Lin, Kee Malesky, Nikki McLay, and Juanisa McCoy.
“Someday in the future my great-granddaughter will ask ‘Google, why do Indians wear a red dot on their foreheads?’ I want the answer to be truly reflective of her ancestry and include the emotions that I would feel in answering that question, rather than the one-size-fits-all answer: ‘It’s common practice to do so’.” - Aprajita Mathur, leading bioinformatics software test engineer.
