A Robust Authentication System Handwritten Documents using Local Features for Writer Identification

  • cc icon

    A robust authentication system for handwritten document images carries considerable importance. The authenticity of legal documents, such as wills, signatures, etc., needs to be verified, in order to prevent fraudulent acts. The characteristics of writing style vary from person to person; by analyzing the local features of handwriting, we can possibly identify the writer. In this paper, we introduce a local feature based approach, in which writer-specific characteristics are extracted. The presence of repetitive patterns within handwriting gave us the idea of dividing the writing into a large number of small sub-images. The similarly shaped sub-images are grouped together in classes. The most repetitive patterns are extracted, and the unknown author of the document is identified, using a Bayesian classifier. We tested the system on 50 documents, achieving a successful identification rate of approximately 94%.


    Authentication system , Identification technology , Gesture authentication , Computer security


    Although there is a trend towards modern equipment, such as tablets, along with the concept of a paperless world, handwritten documents still remain important, particularly pertaining to legal matters. Due to this fact, the verification of the authenticity of handwritten text and signatures has been an open area of research. Researchers have attempted to propose robust authentication systems of writer identification for handwritten document images. In contrast to printed text, handwritten text provides us with extra information, such as insight into the personality of the writer. Individual writing styles are usually stable and consistent over a long period of time; thus analyzing them gives us a chance to identify the author of the text. Research has been underway in this field for over a decade now, but the desired results have yet to be achieved. Another issue that arises is that different languages have different inherent features; consequently it is very hard to propose a system that works for all languages.

    The authors in [1] describe the problem of identifying the author of a document as being recurrent; they state that this issue usually arises in a court of law, where there is often the need to verify the authenticity of a document, such as a will. Similar problems arise in the medical field, where the signature on a prescription must be authenticated [2]. This problem also occurs in the case of banks, where signatures need to be checked and verified, for the issuance of cash or other related goods [3]. Another field where this area of research could be utilized is the analysis of ancient document images, for their indexing, as well as information retrieval. The system can be used for text recognition of handwritten documents, as well as applying the principle of system adaptation to the type of writing [4].

    In this paper, we present an authentication system for writer identification of handwriting. The system can also be used for authenticating signatures. The authentication is done offline, and can be enhanced to work for online signature authentication, for example, in using tablet PCs. The objective here is to find similarities between documents with unknown author and documents with known author. Fig. 1 explains the concept in detail. Writer identification is a slightly different concept from the verification model for two handwriting samples; our goal is to determine whether the given samples were written by one author, or by two different people [5].

    The paper is organized as follows: the first section is about the description of some of the state of the art authentication methods of writer identification. It is followed by a brief account of our proposed system, using local features. Experimental results are presented in the following section to highlight the efficiency of our method. Finally, there are the conclusion and perspectives for future improvements to the system.


    Recently, there has been a lot of interest among researchers in proposing authentication systems for handwritten texts and documents. For that purpose, various features have been proposed to categorize writing styles using minute details. These features are local and global, as well as statistical and they are also used in combination with one another to better distinguish two different writers.

    Said et al. [6] in their work have discussed a global approach, where they take each handwriting sample as a texture, and then utilize Gabor filter and co-occurrence matrices to distinguish between them. The authors in [5] differentiate between individual handwriting by extracting twelve macro features, which are retrieved at either the document or paragraph level. They also extract 5120 binary micro features, which are retrieved from a set of ten characters. Testing was carried out using these features and satisfactory results were achieved. The authors continued this research in [7] as well.

    Structural features like height, width, slope, aspect ratio, etc., have been used in [8] to authenticate a handwritten document image. These features are extracted at line level. There are also other papers on writer authentication, based on fractal analysis of text [2, 9]. Fractal compression and decompression is used in these systems. Bensefia et al. [1], using the concept of writer invariants, proposed a writer identification system based on one-toone correspondence between graphemes extracted from two persons’ handwriting. Other features, such as the probability distributions of edge-based directional vectors, have been employed in [10], and the performance of the authentication system is compared with a number of non-angular features.


    Considering the state-of-the-art work in authentication systems of writer identification, most of the work has been focused on the extraction of some characteristics from text, and then matching them based on these features. Our work is based on a similar approach. We extract certain local features that are redundant in a handwritten text, thus allowing us to specify a writer. These features highlight the frequent trends and patterns present in handwriting. We can divide our system into two main stages: the offline training of the authentication system and the writer identification of a test document.

      >  A. System Training

    The training sub-stage is necessary as we have to create a reference base of all the writers by extracting local inherent characteristics from a text. For this, the text is divided into small same-sized windows. The size of the window is adjusted, according to the text size. It must be kept in mind that the window size should be large enough to contain substantial information in itself. For this division, the simplest way is to divide the text into a grid of identically shaped windows. The squares that do not contain Fig. 1. Authentication system architecture. any text are then removed, and only the squares containing some part of the text are retained in the reference base [2]. This division, however, does not smartly position the windows over text. Thus, we implemented an improved version of the windowing technique, where we do a component-by-component textual division. For each CC, we fix the vertical window position and then tilt the window horizontally, so that it covers a stroke in a smart way. While the approach in [2] moves the complete grid horizontally, our approach of doing it for individual windows allows for a better positioning of the windows over the text. This has been demonstrated in Fig. 2 for a stroke and in Fig. 3 for a complete word.

    Once we have divided the image into sub-images, the next step is to categorize these sub-images into similar groups or classes. This is done because we want to see which patterns occur more frequently, and are thus true representatives of a writer’s handwriting style. In order to cluster, two sub-images are compared, using correlation as the measure of similarity. This algorithm does not need to know the number of classes in advance and thus can be easily implemented [1]. A similarity threshold is empirically chosen. The algorithm works as follows: We place the first sub-image as the center of the first class. Then, for the next patterns, we calculate the correlation between the current pattern and the center of each class. If the distance is shorter than the threshold, the pattern is associated with that class; otherwise, it is placed as the center of a new class, thus forming the basis for a new cluster.

    This is done for all sub-images and they are all clustered in different classes (see Fig. 4). Once completed, we retain only those classes that have at least a given number of elements Φ. In our case, we fixed Φ to 5 in the experiments.

    C = {Ci |card(Ci ) ≥ ϕ}

    For each class Ci, instead of keeping all elements, we only keep one representative sub-image Rpr(Ci). It is the element that is relatively closest to all the other elements of its class. Thus, a document D is represented as:


    Fig. 4 highlights the representative patterns of each class extracted from a single document image. Only the classes that occurred frequently, i.e., having at least Φ elements, have been retained.

    The probability of occurrence of each class is then calculated as:


    The covariance matrix is given by:

    Covi = (Xi Mean(Ci))(XiMean(Ci))T

    In the above equation, Xi is a matrix, having columns that are actually the elements of class i. Mean(Ci) represents the arithmetic mean of class i. In this way, a vector is created for each class and the document is represented by this set of vectors.

    Fi = {Rpr(Ci),P(Ci),Covi}

    D = {Fi ,i card(C)}

    For each handwritten textual document, these vectors are calculated and are added to the reference base.

    Once the reference base is created, we can now test the authentication system, by identifying the writer of an unknown document image.

      >  B. Writer Identification

    In this stage, the main goal is to take a sample s of an unknown handwriting, match it to the stored N samples in the reference base, and identify the writer of s from those N samples of known writers [5]. For this to occur, we first need to extract the same features from the test document image. The test document is first of all binarized using a global threshold. This is because the document images we use are high quality documents; a global threshold is sufficient in our case. The text is then divided into small sub-images just like we did during training. The clustering of the sub-images is done and only the redundant patterns are retained. Each class if represented by one sub-image and thus a similar dimension vector of features is created for the test image as well. Once the feature extraction for the test document is complete, we move to the matching stage, for which we have used Bayesian decision theory.

    The theory behind the Bayesian classifier is the ability to specify a decision problem in probabilistic terms. This assumes that all relevant probability terms are known. So we can write the Bayesian decision rule using the posterior probability P(Ci|x), class likelihood probability p(x|Ci), and prior probability P(Ci):

    iBayes = argimax(P(Ci|x) = argimax p(x|Ci)P(Ci)

    The goal of the above equation is to search for a class i that would maximize the probability that a pattern x belongs to class Ci. If we assume that the class conditional probability p(x|Ci) has a Gaussian distribution for each class Ci, we have:


    where, ∝i is the class mean, and we may write:

    lBayes = argimax[log p(x|Ci) + log P(Ci)]

    Considering the law of normal distribution, after dropping constant values, we come to the following expression for maximization:


    For comparison between an unknown text document T with a document D present in our reference base, the following similarity index is defined:


    The writer of T is identified as the writer of D, thus maximizing the similarity index [1].


    The system thus creates an output showing whether the writer of the test document is present in the reference base or not, and if so, the closest match to the test document.


    To evaluate the system on an internationally recognized test set, we used the IAM database [11]. This database contains sample documents of unconstrained handwritten text. The documents in the database have been scanned at a resolution of 300 dpi, and digitized to 256 gray levels. To test the documents, we first binarized the documents using a global threshold of 140. For various tests, the window size was varied as we tested on four different window sizes at a fixed value of Φ. Training was done on 50 writers, one document per writer. For evaluation, a different text image of each writer was used. Overall, the system achieved a 94% identification rate. The 13 × 13 window size generally performed better than the other sizes, as shown in the graph in Fig. 5.


    To check the application of our method in other scenarios, we used the case of signature authentication. Signature authentication is extremely important, especially in a court of law. The objective here is not to propose a solution to the signature verification problem, but rather to see how the proposed features would work on signatures. In comparison with writer recognition, signature verification enjoys the advantage of comparing the same text (signature) of one individual. On the other hand, the disadvantage is that a very limited amount of text per writer is available, making this a challenging task. We have worked on a collection of signatures from 48 different individuals (students) with 15 signatures of each used for training, compared to 15 for testing. We considered only the simple case where there are no forgeries and the data set comprises genuine signatures only.

    Before the extraction of features, the signatures are first normalized. The normalization includes a rotation according to the inertial axis, and then scaling them in a way that ensures that all the signatures are contained in a rectangle of fixed width (fixed to 300 pixels), thus preserving the proportions of the signature.

    Once normalized, we compute the features of each signature and represent the individual by the mean vector of the features computed from his/her 15 samples. For example, feature distributions are computed from 10 signatures from an individual along with the mean distribution. These mean values are computed for each of the features for all the 48 individuals. Each individual can thus be viewed as a class comprising 15 elements (signatures), represented by its mean value.

    To evaluate, we compute the distance of each signature in the test set (15 × 48 = 720 signatures) to each of the 48 classes (individuals). Ninety-one percent of the time, the signatures are assigned to the correct individual in the training set. The performance on signature verification is quantified, as in the case of writer verification, by the equal error rate of 5.88% on the tested set of signatures, as illustrated in Fig. 8. The error is relatively higher in comparison to the state of the art methods in signature verification. But, as we mentioned earlier, the objective of this study was to analyze the usefulness of the features designed for writer recognition (with a relatively larger amount of text per writer available) on signature verification. Considering this fact, the performance can be regarded as good enough.


    In this paper, we presented a robust authentication system for writer identification. The system is based on extracting local window based features from the text and then classifying the local features into clusters. The frequent patterns are retained and used as a reference base for that particular writer. These patterns mainly characterize handwriting, as each writer writes in their own individual way. The results achieved are promising, and have the potential to carry this method forward. We tested the system for signature verification as well and found that in this case, the system could also be employed. We feel that the identification can further be enhanced, by dividing the text into windows in a more adaptive manner. The method also has the potential to be used for other languages, by simply fixing the window size accordingly. If we can add the capability to automatically select a window size, it would better automate our system.

  • 1. Bensefia A., Nosary A., Paquet T., Heutte L. 2002 Writer identification by writer’s invariants [in Proceedings of the 8th International Workshop on Frontiers in Handwriting Recognition] P.274-279 google
  • 2. Seropian A. 2003 Analyse de document et identification de scripteurs google
  • 3. Plamondon R., Lorette G. 1989 “Automatic signature verification and writer identification: the state of the art [Pattern Recognition] Vol.22 P.107-131 google
  • 4. Nosary A., Heutte L., Paquet T., Lecourtier Y. 1999 Defining writer’s invariants to adapt the recognition task [in Proceedings of 5th International Conference on Document Analysis and Recognition] P.765-768 google
  • 5. Srihari S. N., Cha S. H., Arora H., Lee S. 2002 Individuality of handwriting [Journal of Forensic Sciences] Vol.47 P.856-872 google
  • 6. Said H. E. S., Tan T. N., Baker K. D. 2000 Personal identification based on handwriting [Pattern Recognition] Vol.33 P.149-160 google doi
  • 7. Zhang B., Srihari S. N., Lee S. 2003 ;Individuality of handwritten characters [Proceedings of the 7th International Conference on Document Analysis and Recognition] P.1086-1090 google
  • 8. Marti U. V., Messerli R., Bunke H. 2001 Writer identification using text line based features [in Proceedings of the 6th International Conference on Document Analysis and Recognition] P.101-105 google
  • 9. Bouletreau V., Vincent N., Sabourin R., Emptoz H. 1998 Handwriting and signature: one or two personality identifiers? [in Proceedings of the 14th International Conference on Pattern Recognition] P.1758-1760 google
  • 10. Bulacu M., Schomaker L., Vuurpijl L. 2003 Writer identification using edge-based directional features [in Proceedings of the 7th International Conference on Document Analysis and Recognition] P.937-941 google
  • 11. Marti U. V., Bunke H. 1999 A full English sentence database for off-line handwriting recognition [in Proceedings of the 5th International Conference on Document Analysis and Recognition] P.705-708 google
  • [Fig. 1.] Authentication system architecture.
    Authentication system architecture.
  • [Fig. 2.] Position of window on text component.
    Position of window on text component.
  • [Fig. 3.] Division of a word into small sub-images of window size.
    Division of a word into small sub-images of window size.
  • [Fig. 4.] Local features representing a writer.
    Local features representing a writer.
  • [] 
  • [] 
  • [] 
  • [] 
  • [] 
  • [] 
  • [Fig. 5.] Identification authenticity at different window sizes.
    Identification authenticity at different window sizes.
  • [Fig. 6.] Examples of signatures.
    Examples of signatures.
  • [Fig. 7.] Inertial axis of the signature and the redressed signature.
    Inertial axis of the signature and the redressed signature.
  • [Fig. 8.] Receiver operating characteristic (ROC) curve on the signatures. FAR: false accept rate, FRR: false reject rate.
    Receiver operating characteristic (ROC) curve on the signatures. FAR: false accept rate, FRR: false reject rate.