LLM 기반 인공지능 판정자의 평가구조 분석: Feasibility-Viability 기반 스타트업 사업계획서의 정량적 심사 가능성

This study presents an empirical and exploratory investigation of an AI-based judge model applying a Large Language Model (LLM) to the evaluation of business plans for technology-based innovative firms. Using actual business proposals submitted to TIPS, we apply an LLM-based evaluator that scores six assessment items, consisting of three technological feasibility factors and three business viability factors. Because business plans are inherently unstructured documents composed of narratives, tables, and images, a core structuring process was introduced to transform proposal contents into evaluation units interpretable by the LLM. Few-shot prompting was employed to anchor evaluation criteria, and an in-context (unsupervised) assessment was conducted to emulate the judgment format used by human reviewers. The evaluation was conducted in two stages. In Stage 1, baseline predictive performance was examined using equal weights across items. In Stage 2, evaluation scores were recalculated using contribution-based weights derived from item-level associations with selection outcomes. When compared with TIPS results, predictive performance improved from Stage 1 to Stage 2, with the ROC-AUC increasing from 0.6889 to 0.7833, the Brier score decreasing from 0.1361 to 0.1302, and the PR-AUC improving from 0.8932 to 0.9446, indicating enhanced discrimination under class imbalance. These findings suggest that an LLM-based evaluator can differentiate technological feasibility and business viability in a document-based evaluation setting, and that contribution-based weighting improves alignment with observed decision outcomes. As an initial applied study, this research provides empirical support for using LLMs as complementary co-judges to enhance consistency and reliability in startup evaluation systems.