X Server (Desktop) Programming in Assembler
 jeff owens
 @2008 GNU General Public License


CONTENTS
1.   Introduction - what this document covers
2.   Linux Programming Environments
3.   Overview of X programming
4.   Connecting to the X server
5.   Sending X commands
6.   Receiving X data
7.   Speed Considerations
8.   X protocol extensions
9.   Compiling
10   Example program using asmlibx
11.  Example program without asmlibx
12.  Debugging
13.  X tools
14.  Useful Links
15.  Example 1 source
16.  Example 2 source
  
  
1.0 Introduction
----------------

  This document explores assembler programming
  for the Linux x server. It describes building
  a sample program and debugging x programs.


2.0 Linux Programming Environments
----------------------------------

  Linux programs can run in the console, on a
  terminal, or utilize the X server.  Each
  environment is unique and has different features
  available.  A short summary follows:
  
  Console programs talk directly to the Linux
  kernel and most are command line (text) programs.
  The kernel does not supply graphics functions,
  mouse drivers, or audio handlers.  This environment
  is home to a large number of classical unix
  utilities.  It can host very fast and efficient
  programming, has lots of documentation, and can
  be confusing to new programmers.
  
  Terminal programs run in a "terminal" talking
  to the X server. The terminal environment is
  very close the the console environment and most
  console programs can also run on a terminal.
  There are a large number of different terminals
  and each provides different features.  The more
  popular terminals add mouse handling and menus
  to control fonts and other features.
  
  X servers are starting to dominate Linux computing
  and provide a more robust and simpler interface
  for most programming jobs.  The X server provides
  very good handling for keyboards, mice, graphics,
  and adds window control for working with multiple
  programs at the same time.
  
  Which environment is best for portable programs?
  
  Assembler isn't considered portable and very few
  tools assist with portability.  Currently there
  is an library (HLA) that provides some basic
  functions. It supports terminal programming and
  may be useful for simple text programs. There
  are many non-assembler libraries that supply
  portability for X server/Microsoft  and they
  can be called from most assemblers.  Portable
  assemblers include nasm and fasm.     
  
  Unix to Unix portability has been attempted
  on the LinuxAssembly web site, but has not
  got a lot of support.
  
  If portability is a goal the best path is
  probably to work in a HLL such as "C" and use
  one of the portable libraries.

  If the choice is between console/terminal/
  x server programming, the x server interface
  is well standardized and available on other
  operating systems.
  
  Why choose console/terminal programming?
  
  Unix administration and setup is still done
  in the console and on terminals.  Servers and
  other demanding tasks, do not need a X server
  and run from the console. Also, many embedded
  systems or standalone work stations have
  chosen efficient console programming. All these
  programs are textual and work on minimal hardware.
  
  For textual programs needing efficiency, choose
  the console.
  
  Why choose X server programming?
  
  Today most games, applications, browsers, and general
  use software runs under the X server.  This environment
  provides a fast growing tool set and very robust
  programming arena.  It also, is home to many
  bloated and inefficient programs.  Assembler can
  fit into this environment easily, but few tools
  exist today.  It is probably a good choice for
  casual programming.
  
  The largest audience is also on the x desktop
  and the desktop interacts with web and remote
  computers.  Overall it provides the best future
  growth path.
  
  Other considerations for console/terminals
  
  Most Linux assembler information is about terminals
  or the console. There are lots of example programs
  and a few libraries. This area is popular with novice
  programmers, but they tend to move on to other areas.
  The HLL libraries for the console are still maintained,
  but activity is minimal.
  
  Other considerations for X server programming
  
  The X server is mostly the domain of HLL languages
  and libraries. A few programs call the low level
  xlib and 2 or 3 other low level libraries.
  Low level X programming fits assembler but
  design and coding requires a server mind set.
  
  All communication with the X server is via a
  socket and responses to requests or unsolicited
  input can appear at any time.  This makes
  simple one-step-at-a-time programming unworkable.
  Instead, programs are usually event driven.
  
  For speed all X server requests are collected in
  a buffer and send in one large transfer. Responses
  have a sequence number to indicate which request
  they are for, and not all requests have a
  response.  Most programmers will let the various
  libraries handle X server communication, but
  it helps to understand what is happening.
  
  
  
3.0 Overview of X programming
-----------------------------

  One of the largest problems facing programmers, is
  supporting all the different displays, keyboards,
  and other input devices.  The manufactures may
  provide device drivers and often the operating system
  will try to standardize the interface.  Libraries,
  also try to solve this problem by providing standard
  routines.

  A group at MIT decided the best solution was to
  leap frog the needs of programmers and provide
  a standard tool with additional features for
  abstraction and multiprocessing.  This would
  allow displays to be on a remote computer or
  even multiple displays on one computer.

  They also addressed problems with multiple users
  and programs sharing the same display. Next, they
  wrote a low level library to further isolate
  programmers from the details.  Today, most programs
  have gone further and add another library level.

  The picture looks something like this:

  computer hardware
    operating system
      x server
        x library
          HLL library
            applications

  This stack of handlers has produced fairly fast
  programs, but often they end up bloated in size and
  slow to load.

  Assembler programmers can work with the x library
  or with the HLL library, and may prefer doing
  graphic applications using these libraries. For
  non-graphical applications, the assembly programmer
  can easily bypass the standard libraries and
  gain some speed and size advantages.

  This document introduces the tools and information
  needed for talking directly to a x server.

  Another concept to be aware of is abstraction.
  With x servers we have levels of usage:
    display
    desktop
    window
    sub-window

  The display can be a device or file and a computer
  can have multiple displays defined.  Within a display
  we can have multiple desktops.  Each desktop can have
  a set of windows, and windows can have sub-windows.

  Any program can talk to these abstract items and
  request control of resources.  Creating them and finding
  out about them is where things get difficult.


4.0 Connecting to the X server
------------------------------

  In a multiprocessing environment you want
  the display to be available for everyone
  and still resolve conflicts. Also, you want
  to avoid having any process hog the
  communication channels.

  This is accomplished by using a socket that
  any process can connect to.  To keep this
  socket secure a simple handshake process was
  defined.

  Once we have a socket connected, the data
  flows to and from the x server as demand
  requires. We don't have to worry about
  other programs.

  The connection process goes like this:

   1. Select the display.  Usually, we look
      in a programs environment for the variable:
        DISPLAY=0:0
      The 0:0 specifies our target display.
   2. Look for the authorization string.  This key
      will open x server communication and is
      often placed in a local file.
   3. Create a socket and send our key
   4. Read reply with connection information.


5.0 Sending X commands
----------------------

  Commands are sent as packets of information. These
  packets share a standard format and set of rules.
  For portability all packets have lengths in multiples
  of dwords.  The initial packet always contains:

   1 byte - op code
   1 byte - value depends upon op code
   2 bytes- length of this request in dwords

  The rest of the packet depends upon the op code.
  The format of the various op codes is called the
  protocol.  

  Each request has a sequence number associated with it.
  The first request will be "1" and the next "2", etc.
  This number must be kept by the sending function and
  another counter is kept by the x server. This sequence
  number is needed to identify replies.  The next section
  will discuss this further.

  The protocol op codes (request codes) are:

  
AllocColor 84 CreateWindow 1
AllocColorCells 86 ChangeWindowAttributes 2
AllocColorPlanes 87 GetWindowAttributes 3
AllocNamedColor 85 DestroyWindow 4
AllowEvents 35 DestroySubwindows 5
Bell 104 ChangeSaveSet 6
ChangeActivePointerGrab 30 ReparentWindow 7
ChangeGC 56 MapWindow 8
ChangeHosts 109 MapSubwindows 9
ChangeKeyboardControl 102 UnmapWindow 10
ChangeKeyboardMapping 100 UnmapSubwindows 11
ChangePointerControl 105 ConfigureWindow 12
ChangeProperty 18 CirculateWindow 13
ChangeSaveSet 6 GetGeometry 14
ChangeWindowAttributes 2 QueryTree 15
CirculateWindow 13 InternAtom 16
ClearArea 61 GetAtomName 17
CloseFont 46 ChangeProperty 18
ConfigureWindow 12 DeleteProperty 19
ConvertSelection 24 GetProperty 20
CopyArea 62 ListProperties 21
CopyColormapAndFree 80 SetSelectionOwner 22
CopyGC 57 GetSelectionOwner 23
CopyPlane 63 ConvertSelection 24
CreateColormap 78 SendEvent 25
CreateCursor 93 GrabPointer 26
CreateGC 55 UngrabPointer 27
CreateGlyphCursor 94 GrabButton 28
CreatePixmap 53 UngrabButton 29
CreateWindow 1 ChangeActivePointerGrab 30
DeleteProperty 19 GrabKeyboard 31
DestroySubwindows 5 UngrabKeyboard 32
DestroyWindow 4 GrabKey 33
FillPoly 69 UngrabKey 34
ForceScreenSaver 115 AllowEvents 35
FreeColormap 79 GrabServer 36
FreeColors 88 UngrabServer 37
FreeCursor 95 QueryPointer 38
FreeGC 60 GetMotionEvents 39
FreePixmap 54 TranslateCoords 40
GetAtomName 17 WarpPointer 41
GetFontPath 52 SetInputFocus 42
GetGeometry 14 GetInputFocus 43
GetImage 73 QueryKeymap 44
GetInputFocus 43 OpenFont 45
GetKeyboardControl 103 CloseFont 46
GetKeyboardMapping 101 QueryFont 47
GetModifierMapping 119 QueryTextExtents 48
GetMotionEvents 39 ListFonts 49
GetPointerControl 106 ListFontsWithInfo 50
GetPointerMapping 117 SetFontPath 51
GetProperty 20 GetFontPath 52
GetScreenSaver 108 CreatePixmap 53
GetSelectionOwner 23 FreePixmap 54
GetWindowAttributes 3 CreateGC 55
GrabButton 28 ChangeGC 56
GrabKey 33 CopyGC 57
GrabKeyboard 31 SetDashes 58
GrabPointer 26 SetClipRectangles 59
GrabServer 36 FreeGC 60
ImageText16 77 ClearArea 61
ImageText8 76 CopyArea 62
InstallColormap 81 CopyPlane 63
InternAtom 16 PolyPoint 64
KillClient 113 PolyLine 65
ListExtensions 99 PolySegment 66
ListFonts 49 PolyRectangle 67
ListFontsWithInfo 50 PolyArc 68
ListHosts 110 FillPoly 69
ListInstalledColormaps 83 PolyFillRectangle 70
ListProperties 21 PolyFillArc 71
LookupColor 92 PutImage 72
MapSubwindows 9 GetImage 73
MapWindow 8 PolyText8 74
NoOperation 127 PolyText16 75
OpenFont 45 ImageText8 76
PolyArc 68 ImageText16 77
PolyFillArc 71 CreateColormap 78
PolyFillRectangle 70 FreeColormap 79
PolyLine 65 CopyColormapAndFree 80
PolyPoint 64 InstallColormap 81
PolyRectangle 67 UninstallColormap 82
PolySegment 66 ListInstalledColormaps 83
PolyText16 75 AllocColor 84
PolyText8 74 AllocNamedColor 85
PutImage 72 AllocColorCells 86
QueryBestSize 97 AllocColorPlanes 87
QueryColors 91 FreeColors 88
QueryExtension 98 StoreColors 89
QueryFont 47 StoreNamedColor 90
QueryKeymap 44 QueryColors 91
QueryPointer 38 LookupColor 92
QueryTextExtents 48 CreateCursor 93
QueryTree 15 CreateGlyphCursor 94
RecolorCursor 96 FreeCursor 95
ReparentWindow 7 RecolorCursor 96
RotateProperties 114 QueryBestSize 97
SendEvent 25 QueryExtension 98
SetAccessControl 111 ListExtensions 99
SetClipRectangles 59 ChangeKeyboardMapping 100
SetCloseDownMode 112 GetKeyboardMapping 101
SetDashes 58 ChangeKeyboardControl 102
SetFontPath 51 GetKeyboardControl 103
SetInputFocus 42 Bell 104
SetModifierMapping 118 ChangePointerControl 105
SetPointerMapping 116 GetPointerControl 106
SetScreenSaver 107 SetScreenSaver 107
SetSelectionOwner 22 GetScreenSaver 108
StoreColors 89 ChangeHosts 109
StoreNamedColor 90 ListHosts 110
TranslateCoords 40 SetAccessControl 111
UngrabButton 29 SetCloseDownMode 112
UngrabKey 34 KillClient 113
UngrabKeyboard 32 RotateProperties 114
UngrabPointer 27 ForceScreenSaver 115
UngrabServer 37 SetPointerMapping 116
UninstallColormap 82 GetPointerMapping 117
UnmapSubwindows 11 SetModifierMapping 118
UnmapWindow 10 GetModifierMapping 119
WarpPointer 41 NoOperation 127

   
  The fields for each request code are documented at MIT
  (see links). A good description of these requests is
  contained in the book: X protocol Reference Manual, by
  O'Reilly & Associates


6.0 Receiving X data
--------------------

  Some protocol commands have replies and most
  can generate errors. If we have windows, then
  they we can enable various event notifications.

  All received data follows these rules:
  
  1. The initial packet is 32 bytes long and
     has a standard header.  Normal replies
     have a op code of "1" and errors have a
     opcode of "0". Other opcodes are defined
     for events.
  2. If additional information is coming, it's
     size is given.
  3. All replies to requests must provide a
     sequence number

  The initial packet looks like this:

     1 byte - code
     1 byte - (depends upon code)
     2 bytes- sequence number (if reply or error)
     4 bytes- length of additional data to be sent

  The remaining fields depend on the code byte value.
  
7.0 Speed Considerations
------------------------

  Commands can be sent at any time and it is possible
  events can occur at any time. If we collect requests
  and send a large block at one time, a big speed
  increase occurs. The same is true of some event
  processing.

  All x libraries have request batching and they
  provide a "flush" function to force sending the
  current buffer contents.  Normally, requests
  are sent when the buffer is full of requests.


8.0 X protocol extensions
--------------------------


  Extensions to protocol (requests) can be added
  by compiling them into x servers.  The programmer
  can use these extensions as follows:

  1. Use QueryExtension request to check if
     extension is available.
  2. Get extension opcode from reply to
     QueryExtension.

  The example program in section 10.0 will list
  available extensions.

  
9.0 Compiling
-------------

  Our example code is written for the nasm assembler
  and can be compiled with:

  nasm -f elf -O999 -g -o example.o example.asm

  where: -f elf  <-- output in elf format
         -O999   <-- optimization passes
         -g      <-- include debug information
         -o example.o <-- create object file
         example.asm  <-- assembler source file

  The example.o output file must be linked with
  two libraries to create an executable.

  ld -o example example.o asmlibx.a asmlib.a

  where: -0 example   <-- output executable
         example.o    <-- input file to link
         asmlibx.a    <-- library to link with
         asmlib.a     <-- library to link with

  The linker "ld" is on almost all Linux distributions
  and is part of the bin utilities. Nasm is available
  at sourceforge (see links).
  
10.0 Example program - using asmlibx
-------------------------------------

  The full source for example is included
  in section 15.  A discussion of the code
  follows:

  Once we are connected, extensive information
  can be requested from the server.
  For a starter, we might want to know which
  extensions this server supports.  For this
  we need to send a protocol request:

  ListExtensions - protocol #99

  If we utilize the library "asmlibx" our program
  would appear as follows:

 ---- program start -----
_start:
  call  env_stack       ;save ptr to environment
  call  x_list_extension;get x server extensions
  call  show_extensions ;display extensions
 
  mov   eax,01          ;exit function
  int   byte 80h        ;exit program
 ---- program end ----

  The call to env_stack creates a global variable
  that points to the program environment strings.
  This is used by a server connection routine to
  find our display number.

  The x_list_extension call first checks to see if
  we are connected, and if not, attempts a connection.
  Next, it sends a request to server and collects
  the replies.

  Finally, the show_extensions is a small subroutine
  we must write to display the extension names.

  The complete listing of this program can be
  found in section 14.0.
  
  That was easy, but what happened?  Most of the
  details were handled by asmlibx and we are
  left with the results.  What if we wanted to
  go down another level?

  The next example goes into the gory details.
  If you want to enter example 1 and test it,
  skip to section 12 Debugging.

11.0 Example program - without asmlibx
--------------------------------------
 
  This example (example2) does not use any libraries and is
  identical to example 1. It is interesting to note that the
  source file for example 1 is about 600 bytes and this example
  has a source file of over 32,000 bytes. Libraries save a lot of
  coding time and effort!

  The easiest way to approach x programming is to use
  a common x connect routine for all programs.  Then
  use the common send/receive handlers to talk to
  the x server. The server protocol functions may
  be available in  asmlibx, but often they must be
  coded.

  In example 2 we will talk about how to code a
  protocol function and use asmlibx to send
  and receive data.

  The library function that connects to a x server
  is called x_connect. We can call x_connect at
  the start of a program or let the send function
  do it for us. Our only requirement is to call
  env_stack at the start of our program.

  The two library functions for sending and receiving
  are:

    x_send_request
    x_wait_reply

  We provide x_send_request with a protocol packet
  then call x_wait_reply to get the response.  The
  code to get a list of x server extensions would
  look like this:
     
    list_extension_request:
     db 99      ;opcode
     db 0       ;unused
     dw 1       ;request length in dwords
    qer_end:
    
    x_list_extension:
      mov       ecx,list_extension_request
      mov       edx,(qer_end - list_extension_request)
      neg       edx             ;indicate reply expected
      call      x_send_request
      js        ger_exit
      call      x_wait_reply
    ger_exit:
      ret

  If a reply is expected we must negate the size of
  the send packet. The tells the send function that
  a flush is needed after sending the packet.  If we
  didn't flush, the packet would sit in a buffer and
  the reply would return a timeout error.

  If all goes well, the reply pointer will be returned
  in register ecx.

   resb 1  ;1 Reply
   resb 1  ;number of names returned
   resb 2  ;sequence number
   resb 4  ;reply length
   resb 24 ;unused
   resb 1  ;length of extension n
   resb x  ;extension n string

   resb 1  ;length of extension n+1
   resb x  ;extension n+1 string

  The length of the reply packet is variable
  and we must check the name count and
  use it for processing.  Also, this reply
  is held in a temporary buffer and the
  data must be used before other library
  functions are called.

  We now have the tools to code complex programs
  by adding our own protocol functions.   All we  need
  is to connect and then use x_send_request and
  x_wait_reply .   If a library function exists  we could
  use it, but mixing libraries may not  work.  Each
  library would try to set up  its own x server connection.


12.0 Debugging
--------------

  To quickly check if the x server communication
  is occurring, we can use "tracex".  Tracex creates
  a log file containing all server communication
  packets.  Run tracex as follows:

  tracex example1

  The output trace is stored at example1.tra.
  Entries contain the name of protocol functions,
  sequence numbers, and a dump of packet contents.
  We should see the following:

  --- start of file example1.tra ---  
   Authorization packet written
   
   ListExtensions request#=0001
   63 00 01 00                                       c...
   
   -Event-reply-to#0001
   01 1D 01 00 50 00 00 00 04 00 00 00 01 00 00 00   ....P...........
   00 00 00 00 04 00 00 00 90 01 00 00 04 00 00 00   ................
   
   -continuation of reply#0001
   05 53 48 41 50 45 16 4D 49 54 2D 53 55 4E 44 52   .SHAPE.MIT-SUNDR
   59 2D 4E 4F 4E 53 54 41 4E 44 41 52 44 0C 42 49   Y-NONSTANDARD.BI
   47 2D 52 45 51 55 45 53 54 53 04 53 59 4E 43 10   G-REQUESTS.SYNC.
   4D 49 54 2D 53 43 52 45 45 4E 2D 53 41 56 45 52   MIT-SCREEN-SAVER
   07 58 43 2D 4D 49 53 43 18 58 46 72 65 65 38 36   .XC-MISC.XFree86
   2D 56 69 64 4D 6F 64 65 45 78 74 65 6E 73 69 6F   -VidModeExtensio
   6E 0C 58 46 72 65 65 38 36 2D 4D 69 73 63 0B 58   n.XFree86-Misc.X
   46 72 65 65 38 36 2D 44 47 41 04 44 50 4D 53 07   Free86-DGA.DPMS.
   54 4F 47 2D 43 55 50 1B 45 78 74 65 6E 64 65 64   TOG-CUP.Extended
   2D 56 69 73 75 61 6C 2D 49 6E 66 6F 72 6D 61 74   -Visual-Informat
   69 6F 6E 06 58 56 69 64 65 6F 0A 58 2D 52 65 73   ion.XVideo.X-Res
   6F 75 72 63 65 0D 44 4F 55 42 4C 45 2D 42 55 46   ource.DOUBLE-BUF
   46 45 52 03 47 4C 58 07 53 47 49 2D 47 4C 58 06   FER.GLX.SGI-GLX.
   52 45 43 4F 52 44 07 4D 49 54 2D 53 48 4D 0F 58   RECORD.MIT-SHM.X
   49 6E 70 75 74 45 78 74 65 6E 73 69 6F 6E 05 58   InputExtension.X
   54 45 53 54 09 58 4B 45 59 42 4F 41 52 44 0B 58   TEST.XKEYBOARD.X
   43 2D 41 50 50 47 52 4F 55 50 08 53 45 43 55 52   C-APPGROUP.SECUR
   49 54 59 06 58 46 49 58 45 53 0F 58 46 72 65 65   ITY.XFIXES.XFree
   38 36 2D 42 69 67 66 6F 6E 74 06 52 45 4E 44 45   86-Bigfont.RENDE
   52 05 52 41 4E 44 52 06 44 41 4D 41 47 45 00 00   R.RANDR.DAMAGE..
  --- end of file example1.tra ---

  The second entry is our request.  It has a op code
  of 63 hex or 99 decimal. The length field says it
  is one dword long.

  Following the request is a reply.  The reply code
  of 01 says we were successful and the length field
  of 50 says we need to read an additional packet of
  80 decimal bytes.
 
  If we want to dig a little deeper, the asmbug program
  will walk through the program as it executes.  Type:

  asmbug example1

  AsmBug is easy to use if you just assume it knows what
  you want.  Click on a address to set a break and then
  click run.  Or click step and watch it execute.

  If you want a complete trace of every instruction, the
  asmtrace program can be used.  It is interactive and
  also creates the log file example1.tra. Beware, asmtrace
  produces a lot of data and expect a large file.

  The final debugging tool is inserting print statements
  into our example program.  Often this works well with
  event driven programs.  Our simple example program will
  not need print statements, but it is good to be prepared.
  The asmlib provides some functions to do logging which
  save registers to avoid introducing additional bugs.

  links to tracex, asmbug, and asmtrace are in section 14
  
13.0 X tools
------------

  For graphics, sound, and other needs, the HLL
  libraries may be of interest.  Libraries to
  explore are: GTK, QT, and SDL.

  xwininfo

  To get information about displayed windows, run
  the xwininfo program. It allows you to click on
  window of interest and then sends a lot of data
  to stdout.

  xdpyinfo

  A good display of x server information can be
  obtained from xdpyinfo.  Just type: xdpyinfo
  and stand back.  It will dump just about everything.


14.0 Useful Links
-----------------

link to home of this tutorial and asmlib, asmlibx,
asmbug, tracex, and other x related programs
  http://linuxasmtools.net/x/

link to hundreds of X related web sites
  http://www.rahul.net/kenton/xsites.html#FAQ

nasm assembler
  http://sourceforge.net/projects/nasm

bin utilities, ld and others
  http://www.gnu.org/software/

individual tools
  http;//sourceforge.net/projects/tracex
  http://sourceforge.net/projects/asmbug
  http://sourceforge.net/projects/asmtrace

libraries
  http://sourceforge.net/projects/asmlibx
  http://sorceforge.net/projects/asmlib

X documentation, protocol, extensions
  http://webcvs.freedesktop.org/xorg/xc/

15.0 example 1 source
---------------------

A source file for this example is in download page
of http://linuxasmtools.net/x/

;-code1-
;-code end-
   
16.0 example 2 source
---------------------

A source file for this example is in download page
of http://linuxasmtools.net/x/

;-code2-
;-code end-