생성AI의 기만수단으로의 오용: 실태, 대응기술, 법적 쟁점

As generative AI applications become more widespread, so do their misuses in the form of deepfakes and other deceptive synthetic media made to impersonate real people, counterfeit or plagiarize authentic works, and disseminate disinformation. This paper explores how law enforcement and AI companies can play different roles and carry varying degrees of responsibility to contain AI abuses in society. Law enforcement must first apply existing laws to protect entities such as consumers and investors as well as social structures such as the electoral process. Lawmakers should also adapt the current legal system to address targeted concerns arising from deepfakes such as spear-phishing or astroturfing. AI companies at various points in the supply chain are also expected to partake in this effort. For instance, foundation model developers can integrate mitigating technologies like watermarking, while deployers and platform services can adapt filtering or detection tools. Here, the extent of responsibility borne by foundation model suppliers is still a point of contention – under the EU AI Act, foundation model suppliers are obligated to monitor downstream usages for “reasonably foreseeable misuse.” It is also crucial to be mindful of over-regulating the industry. Excessive legal obligations could restrict downstream usages, hinder freedom of expression, discourage model open-sourcing, and introduce other variations of market interference in application markets. Hence, this paper recommends law enforcement to facilitate model suppliers to self-regulate as in the U.S. or to implement incentive schemes such as safe harbor provisions. As a minimum safeguard, the providers, upon notice of deceptive output, should be prepared to respond promptly by shutting down or limiting API access, disclosing investigative information, and otherwise addressing requests from affected parties.