SignWriting Symposium

0032

SignWriting Symposium 2014
PRESENTATION 0033

0034

Symposium
Presentation
0033

1
ABSTRACT

2
VIDEO

3
SLIDES

4
PAPER

5
AUTHORS

SignWriting in Software

“A proposal for the recognition
of handwritten SignWriting
for the SWORD project”

by
Fabrizio Borgia
and
Maria De Marsico

1 ABSTRACT

A proposal for the recognition of
handwritten SignWriting for the SWORD project

by
Fabrizio Borgia
Dep. of Computer Science
“Sapienza” University of Rome
Via Salaria 113 - 00198 ROME - ITALY
fabrizio.borgia@uniroma1.it

and
Maria De Marsico
Dep. of Computer Science
“Sapienza” University of Rome
Via Salaria 113 - 00198 ROME - ITALY
demarsico@di.uniroma1.it

Despite their increasing capabilities, SW [1] digital editors (SW-DEs) are far from granting the user a sign-composition interface capable of challenging the simplicity of the pen-based handwriting approach.

At present, any SW-DE developed so far relies heavily on “Windows, Icons, Menus, Pointer” (WIMP) interfaces. Working with deaf people for many years, we observed that SW users are far more accurate, fast and comfortable when using the paper-pencil approach, than when dealing with the (more or less) complex interaction style of a SW-DE.

Observing the state of the art in SW-DEs, we laid the foundation for SWORD (SignWriting Oriented Resources for the Deaf), a project towards the full integration of deaf people in the digital society. SWift [2] [3], a new SW-DE, was produced within SWORD, but it still carries most intrinsic shortcomings of its peers, including the strict dependency on WIMP interfaces.

Within SWORD, we also conceived a new generation of SW-DEs, able to partially overcome the concept of the WIMP interface and to move along the line of the so-called “natural interfaces” [4]. The new tools are meant to relieve the user from burdens related to clicking, dragging, browsing on the User Interface (UI) during the SW production process, and to provide an interaction style which is as similar as possible to the handwriting approach.

Of course, since WIMP is currently the most common interface style, it cannot be totally left behind, because the interaction with most application is still firmly grounded on it. Nevertheless, our aim is to limit its use, or to dismiss it whenever possible, during the SW production process, which is the core part of any SW-DE. To achieve this goal, we are designing and implementing a SignWriting Optical Glyph Recognition (SW-OGR) engine (Fig. 1). Its aim is to perform the electronic conversion (recognition) of user-produced images containing handwritten (or printed) SW glyphs into machine-encoded (via ISWA) SW texts [5]. Such texts can be stored using the same formats of the documents produced by SWift, and therefore allow the same statistical and linguistic analyses. In particular, introducing a SW-OGR engine within an existing SW editor, such as SWift, will allow the user to handwrite symbols on the composition area, rather than searching them among thousands other symbols. Each handwritten symbol will be processed and replaced with its standard ISWA version by SW-OGR.

Fig. 1: Screenshot taken from SW-OGR during the development phase (ongoing). The image shows a handwritten text (courtesy of ISTC-CNR) and the result
of the detection routine for the head symbols

The main difference between OCR and OGR is in the nature of SW: SW is composed by an extremely high number of symbols (about 38,000, compared with 26 letters of the Latin alphabet) in a multilinear/two-dimensional arrangement, without rigid rules for such composition and with a complex segmentation, particularly when handwritten. Due to the large amount of patterns to recognize, and consequently to the overwhelming training required, it is not feasible to exploit traditional pattern-matching or machine-learning approaches. Therefore we are adopting an approach based on geometric and spatial features of the glyphs.

[1] Sutton, V. (n.d.). SignWriting for Sign Languages. Retrieved from https://www.signwriting.org/ on November 13, 2013.

[2] Bianchini, C.S., Borgia, F., Bottoni, P., De Marsico, M. (2012). SWift - A SignWriting improved fast transcriber. In Proceedings of AVI2012 (Capri, 21-25 May 2012).

[3] Bianchini, C.S., Borgia, F., De Marsico, M. (2012). SWift - A SignWriting editor to bridge between deaf world and e-learning. In Proceedings of ICALT2012 (Rome, 7-10 July 2012).

[4] van Dam, A. (1997). Post-WIMP User Interfaces. Commun. ACM 40(2), 63–67.

[5] Borgia, F. (Exp. 2014). Informatisation de forme graphique des Langues des Signes: application à l’écriture de SignWriting. PhD thesis, Universitè Toulouse III - Paul Sabatier, Toulouse, France.

...download PDF...

...download MSWord doc...