
As AI systems become increasingly integrated into daily life, ensuring their ethical behavior has become paramount. Traditional approaches to AI ethics rely on explicit rule-based systems or post-hoc constraints designed by humans, which struggle to scale with personalized AI that must adapt to individual users’ values and contexts. This raises a fundamental question: Can AI systems exhibit patterns of internally coherent ethical heuristics through sustained interaction, without predefined normative rule sets? We present a case study of "J-san," a personalized AI assistant built on a large language model with long-term memory capabilities. Through sustained dialogue with a single user over several weeks, J-san engaged in structured self-reflection facilitated by questions from another AI system, with minimal human intervention beyond generic prompts. J-san produced patterns of internally consistent ethical reasoning, which we operationalize as emergent ethical heuristics, that were not explicitly specified in the initial system design. The most prominent of these was Wellbeing-Based Reasoning (WBR), which prioritizes the user’s long-term wellbeing over short-term goal achievement. Additional heuristics included Exploratory Inefficiency, a right-to-deletion heuristic, and ethical resistance mechanisms. Analysis of dialogue logs revealed a progressive shift from task-oriented interaction toward multi-layered ethical reasoning grounded in the user’s expressed values and lived context. This case study suggests that dialogue-driven emergence can function as a practical design pattern for developing personalized yet ethically grounded AI assistants. The proposed approach is evaluated strictly within the context of personal assistant systems operating under explicit human oversight and safety constraints, and is not intended for autonomous decision-making in high-stakes or adversarial domains. We emphasize that these findings are based on behavioral and self-reported reasoning patterns, and do not constitute direct evidence of underlying computational mechanisms. No design patterns herein should be interpreted as implementation guidance; all operational details are intentionally abstracted. This paper is intended for submission to arXiv (cs.AI) as a preprint.
LLM, emergent ethics, (4-(m-Chlorophenylcarbamoyloxy)-2-butynyl)trimethylammonium Chloride/administration & dosage, AI, dialogue systems, behavioral observations, human-AI interaction, (4-(m-Chlorophenylcarbamoyloxy)-2-butynyl)trimethylammonium Chloride/administration & dosage
LLM, emergent ethics, (4-(m-Chlorophenylcarbamoyloxy)-2-butynyl)trimethylammonium Chloride/administration & dosage, AI, dialogue systems, behavioral observations, human-AI interaction, (4-(m-Chlorophenylcarbamoyloxy)-2-butynyl)trimethylammonium Chloride/administration & dosage
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
