Unsupervised Learning for Fraud Detection in Cooperative Transactions: A Hierarchical Clustering and One-Class SVM Approach

Anomaly detection in financial transactions is a critical challenge for mitigating risks associated with fraudulent activities. This work proposes a robust framework tailored to the cooperative financial sector, leveraging unsupervised learning techniques to detect anomalous behavior in transaction data. Hierarchical clustering is first applied to segment users into distinct behavioral groups, enabling the identification of patterns indicative of both normal and suspicious activity. Subsequently, a One-Class Support Vector Machine (SVM) is employed for anomaly detection, achieving Area Under the Curve (AUC) scores of up to 0.90 in clusters with well-defined transactional patterns. Visualization using t-distributed Stochastic Neighbor Embedding (t-SNE) reveals minimal overlap between normal and anomalous transactions within these clusters, further demonstrating the effectiveness of the model. However, performance decreases in clusters exhibiting higher behavioral variability, highlighting the challenges inherent to diverse financial data. This research underscores the importance of domain-specific anomaly detection strategies and offers practical insights for enhancing fraud prevention in financial systems-particularly within underrepresented sectors such as cooperative banking. Future work will focus on improving scalability and model interpretability to enhance applicability in real-world operational environments.