This project is an attendance system that transfer handwritten names into strings can be stored in Excel sheets or you can modify it to be recorded in Databas. Also user can attach a photo for students to count the actual number of attendants.
1. HandwrittenTextRecognition trained by I AM Handwriting Dataset
We had a great challenge in this model as the results were not applicable to be used to search in a database or Excel sheet due to missing or changing any letter of the handwritten name or any wrong spaces.
So we did the following :
We used a scoring function to make a unique score for every name predected from the model and we combared it with the scores from the Excel sheet to make the matching.
2. Detectron2 Human KeyPoints Detection trained by COCO-Dataset
Detectron2 is Facebook AI Research's next generation software system that implements state-of-the-art object detection algorithms.
line segmentation is done through pre-processing, feature extraction and segmentation. Line Segmentation is used to identify the lines present in the paragraph. This is important as many people have a tendency to not write in a straight line.
he final model is the handwriting recognition model which takes a line as input and converts the line into digital text. This model consits of a CNN-biLSTM architecture. The loss used is the CTC (Connectionist Temporal Classification) loss.
Here is the CNN-biLSTM architecture model.
The input lines are sent into the CNN to extract features from similar patterns. These image features are then sent to a sequential learner which are the bidirectional LSTMs which are then sent to the output string that predict the character based on the alphabet with the highest predicted value given by the model.
We use detectron2 for checking the number of the people in the place to make sure there's no way to cheat. We achieve that through taking an photo or video stream of a camera(webcam or phone camera) and return number of person instances in this photo.
- mxnet
- pandas
- matplotlib
- numpy
- skimage
- Linux or macOS
- Python ≥ 3.6
- PyTorch ≥ 1.3
- torchvision that matches the PyTorch installation. You can install them together at pytorch.org to make sure of this.
- OpenCV, optional, needed by demo and visualization
- gcc & g++ ≥ 4.9
- openpyxl
- difflib
Word segmentation parameters
Word recognetion parameters
# install dependencies: (use cu101 because colab has CUDA 10.1)
pip install -U torch==1.5 torchvision==0.6 -f https://download.pytorch.org/whl/cu101/torch_stable.html
pip install cython pyyaml==5.1
pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'
pip install detectron2==0.1.2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/index.html
cd detectron2; python setup.py install