W3C home > Mailing lists > Public > semantic-web@w3.org > November 2016

Moral Reasoning Systems

From: Adam Sobieski <adamsobieski@hotmail.com>
Date: Mon, 21 Nov 2016 19:33:11 +0000
To: "semantic-web@w3.org" <semantic-web@w3.org>
Message-ID: <BY2PR01MB18807F4EF84E73341E411E37C5B50@BY2PR01MB1880.prod.exchangelabs.com>
W3C Semantic Web Interest Group,

I would like to broach with you the interesting topics of automated moral reasoning.


Automated reasoning is a branch of artificial intelligence dedicated to understanding different aspects of reasoning; moral reasoning is reasoning concerned with morality. Automated moral reasoning is a research topic pertaining to the understanding of and the modeling and simulation of moral reasoning.

Moral Reasoning Systems and Education

Five varieties of moral reasoning systems with educational applications to consider are indicated.

Firstly, there is a variety of moral reasoning system with console-based or text-based user interfaces, a variety which possibly makes use of custom programming languages. This variety requires some specialized expertise to use, resembling, perhaps, computer algebra systems, automated theorem provers and proof assistants.

Secondly, there is a variety of moral reasoning system which interoperates with software applications requiring less specialized expertise to use, software where the users needn’t be computer programmers. Examples include decision support systems, software which support individual or organizational decision-making activities.

Thirdly, there is a variety of moral reasoning system with natural language and multimodal user interfaces. This variety includes dialog systems, virtual humans, intelligent personal assistants and intelligent tutoring systems. This variety can conveniently answer, discuss and advise larger numbers of users with regard to questions that they might ask, including in educational contexts.

Fourthly, there is a variety of moral reasoning system which interoperates with the processing and generation of stories, fables, parables or exemplums. This variety can be of use in processing the moral messages of literary texts and generating literary texts which teach moral messages.

Fifthly, there is a variety of moral reasoning system which interoperates with interactive digital entertainment, serious games, simulations and learning environments. This variety interoperates with virtual interactive storytellers, virtual directors, drama managers, experience managers and other educational narrative technologies.

Automated Moral Reasoning and Planning

Automated planning and scheduling is a branch of artificial intelligence concerned with the realization of strategies or action sequences. Planning algorithms are often instrumental to generating the behavior of intelligent systems and robotics.

Machine ethics, or computational ethics, is a part of the ethics of artificial intelligence concerned with the moral behavior of artificially intelligent systems. Moral reasoning components should be interoperable with planning and scheduling components.

Uses of planning are much broader than robotics. Uses of planning extend into every sector, into industry, academia, science, military and government, and into public policy. Combinations of planners and moral reasoning can provide societal benefits transcending robotics and machine ethics.


Moral reasoning systems can provide broad societal benefits including computer-aided moral reasoning, computer-aided authoring of literature, new tools for philosophy, law, social sciences, the digital humanities, new decision support and public policy technologies, and new tools for education.


Barber, Heather, and Daniel Kudenko. "Generation of Adaptive Dilemma-based Interactive Narratives." IEEE Transactions on Computational Intelligence and AI in Games 1, no. 4 (2009): 309-326.

Colyvan, Mark, Damian Cox, and Katie Steele. "Modelling the Moral Dimension of Decisions." Noûs 44, no. 3 (2010): 503-529.

French, Simon. Decision Theory: An Introduction to the Mathematics of Rationality. Halsted Press, 1986.

Goldin, Ilya M., Kevin D. Ashley, and Rosa L. Pinkus. "Introducing PETE: Computer Support for Teaching Ethics." In Proceedings of the 8th international conference on Artificial intelligence and law, pp. 94-98. ACM, 2001.

Greco, Salvatore, J. Figueira, and M. Ehrgott. "Multiple Criteria Decision Analysis." Springer's International series (2005).

Harmon, Sarah. "An Expressive Dilemma Generation Model for Players and Artificial Agents." In Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference. 2016.

Hodhod, Rania. "Interactive Narrative and Intelligent Tutoring for Ill-Defined Domains." (2008).

Hodhod, Rania, and Daniel Kudenko. "Interactive Narrative and Intelligent Tutoring for Ethics Domain." Intelligent Tutoring Systems for Ill-Defined Domains: Assessment and Feedback in Ill-Defined Domains. (2008): 13.

Hodhod, Rania, Daniel Kudenko, and Paul Cairns. "AEINS: Adaptive Educational Interactive Narrative System to Teach Ethics." In AIED 2009: 14th International Conference on Artificial Intelligence in Education Workshops Proceedings, p. 79. 2009.

Hodhod, Rania, Daniel Kudenko, and Paul Cairns. "Serious Games to Teach Ethics." AISB'09: Artificial and Ambient Intelligence (2009).

Lapsley, Daniel K. Moral Psychology. Westview Press, 1996.

Mancherjee, Kevin, and Angela C. Sodan. "Can Computer Tools Support Ethical Decision Making?." ACM SIGCAS Computers and Society 34, no. 2 (2004): 1.

McLaren, Bruce M. "Extensionally Defining Principles and Cases in Ethics: An AI Model." Artificial Intelligence 150, no. 1 (2003): 145-181.

McLaren, Bruce M. "Computational Models of Ethical Reasoning: Challenges, Initial Steps, and Future Directions." IEEE intelligent systems 21, no. 4 (2006): 29-37.

Prakken, Henry, and Giovanni Sartor. "Law and Logic: A Review from an Argumentation Perspective." Artificial intelligence 227 (2015): 214-245.

Rahwan, Iyad, Simon D. Parsons, and Nicolas Maudet. Argumentation in Multi-agent Systems. Springer-Verlag Berlin Heidelberg, 2010.

Robbins, Russell W., William A. Wallace, and Bill Puka. "Supporting Ethical Problem Solving: An Exploratory Investigation." In Proceedings of the 2004 SIGMIS conference on Computer personnel research: Careers, culture, and ethics in a networked environment, pp. 134-143. ACM, 2004.

Saptawijaya, Ari, and Luís Moniz Pereira. "Towards Modeling Morality Computationally with Logic Programming." In International Symposium on Practical Aspects of Declarative Languages, pp. 104-119. Springer International Publishing, 2014.

Schrier, Karen. "EPIC: A Framework for Using Video Games in Ethics Education." Journal of Moral Education 44, no. 4 (2015): 393-424.

Sharipova, Mayya, and Gordon McCalla. "Supporting Students’ Interactions over Case Studies." In International Conference on Artificial Intelligence in Education, pp. 772-775. Springer International Publishing, 2015.

Tappan, Mark B., and Lyn Mikel Brown. "Stories Told and Lessons Learned: Toward a Narrative Approach to Moral Development and Moral Education." Harvard Educational Review 59, no. 2 (1989): 182-206.

Tappan, Mark B. "Hermeneutics and Moral Development: Interpreting Narrative Representations of Moral Experience." Developmental Review 10, no. 3 (1990): 239-265.

Tappan, Mark B., and Packer, M. (Eds.). Narrative and Storytelling: Implications for Understanding Moral Development. New Directions for Child Development, #54. San Franciso Jossey-Bass, 1991.

Vitz, Paul C. "The Use of Stories in Moral Development: New Psychological Reasons for an Old Education Method." American Psychologist 45, no. 6 (1990): 709.



Best regards,
Adam Sobieski
Received on Monday, 21 November 2016 19:33:46 UTC

This archive was generated by hypermail 2.3.1 : Monday, 21 November 2016 19:33:50 UTC