1. Goal of the Project
For a given Roman coin image, the goal is to

1. to automatically find visual characteristics of the coin which make it distinguishable from the others
2. to identify Roman Imperial Coinage (RIC) label of the coin

Figure 1. Sample obverse and reverse images of two an- cient Roman imperial coins. Both coins depict the same emperor (Domitian) on the obverse side but have distinct reverse depictions, resulting in different Roman Imperial Coinage (RIC) labels. The descriptions for them are (a) obverse: Laureate head right, Reverse: Minerva standing right on capital of rostral column with spear and shield to right owl, and (b) obverse: Laureate head right, Reverse: Pegasus right.

2. Motivation and Practical Applications

• Understanding the ancient Roman coins could serve as references to understand the Roman empire
• A reliable and automatic method to recognize the coins is necessary as the coin market is very active and many people are collecting coins as hobby
• Ancient coins are becoming subject to a very large illicit trade. Recognition of the ancient Roman coins is not easy for novices but requires knowledge.

3. Coin Classification Using Convolutional Neural Networks (CNNs)

• Fine-tuning on pre-trained CNN models
  

  – Considering the number of the coin images in our dataset (about 4500), the CNN model is likely to be under-fitted if we train it only on the coin dataset even if we use the data augmentation method
  – We adopt one of the most popular architecture proposed by Krizhevsky et al. [2] which is pre-trained on the ImagetNet with millions of natural images

• Hierarchical classification
  

  – One Roman emperor includes several RIC labels as shown in Figure 1 while one RIC label belongs to exactly one emperor. Therefore, we can build a tree structure to represent the relationship between the Roman emperors and the RIC labels as depicted in Figure 2
  – Then the final probability is defined to be the product of the probabilities on the path from the root to the leaf as \[p(e|I_o) \cdot p(r|I_r) \cdot \delta(Pa(r) = e)\] where \(Pa(r)\) is the parent of node \(r\) and \(\delta(\cdot)\) is the indicator function

  

  Figure 2. Hierarchical classification for the RIC label

4. Finding Characteristic Landmarks on Roman Coins

• Characteristic region set: the smallest set of local patches to represent the identity of the full image
• Representation of the masked image

\[ f_\mathbf{I}(\mathbf{x}) = \left(\sum_{k=1}^{K} x_k \cdot \mathbf{I}_k\right) \otimes \mathbf{C} \] where \(\mathbf{I}\) is an image, \(\mathbf{I}_k\) the k-th subregion of image \(\mathbf{I}\), \(x_k \,\,(0 \leq x_k \leq 1)\) a transparency weight for the k-th subregion (0: transparent, 1: full intensity), \(\mathbf{C}\) a normalization vector

• Objective function: finding an image which consists of the smallest set of regions but still can be correctly classified

\[ \begin{align*} \min_\mathbf{x} & \quad \ell_c\left(f_\mathbf{I} (\mathbf{x})\right) + \lambda \mathcal{R}(\mathbf{x}) \\ \mbox{s.t.} & \quad p\left(c|f_\mathbf{I}(\mathbf{x})\right) > p\left(c|f_\mathbf{I}(\mathbf{1})\right) – \epsilon,\\ & \quad \epsilon > 0, \nonumber \end{align*} \] where \(\ell_c(\cdot)\) is the loss function of the CNN model, \(\mathcal{R}(\mathbf{x})\) a regularization function and \(\lambda\) is a hyper parameter to control the regularization, and we place the constraint so that the prediction probability of the masked image \(f_\mathbf{I}(\mathbf{x})\) may differ from the original image \(f_\mathbf{I}(\mathbf{1})\) at most \(\epsilon\)

5. Coin Data Set

• Coin images are collected from a numismatic website
• 4526 Roman Imperial coins with RIC 314 labels and 96 Roman emperors

– Annotated for visual analysis (the original dataset only has numismatic annotation)
– Both obverse and reverse images for each coin
– Each emperor has at least 10 coins

• High resolution images : at least 300-by-300 pixels

6. Experimental Results

• Classification
  – Baseline method: SVM

  

  Table 1. Classification Accuracies for SVM and CNN

  

  Figure 3. Confusion matrices of CNN and SVM

• Finding Landmarks



Figure 4. Visualization of landmarks as a function of \(\epsilon\)


Figure 5. Discovered landmarks for obverse and reverse images

7. Conclusion

In this project, we proposed a novel method to discover the characteristic landmarks of the ancient Roman imperial coins. Our method automatically finds the smallest set of the discriminative regions sufficient to represent the identity of the full image and distinguish it from other available classes.

The qualitative analysis on the visualization of the discovered regions confirm that the proposed method is able to effectively find the class-specific regions but also it is consistent with the human expert annotations. The proposed framework to identify the ancient Roman imperial coins outperforms the previous approach in the domain of the coin classification by using the hierarchical structure of the RIC labels.

References

• [1] J. Kim and V. Pavlovic. “Discovering Characteristic Landmarks on Ancient Coins using Convolutional Networks”. 2015. Available: http://arxiv.org/abs/1506.09174.
• [2] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks”. NIPS. 2012