Forwarded mail...of buzzwords. Anyway, over the years I have alot of experience
John Marshall
john_w_marshall@palm.com
Thu Mar 16 01:29:00 GMT 2000
>From Vivek_Magotra@palm.com Thu Mar 16 01:23:20 2000
Return-Path: <Vivek_Magotra@palm.com>
Received: from localhost (IDENT:johnm@localhost [127.0.0.1])
by kovalevskaya.palm.com (8.9.3/8.8.7) with ESMTP id BAA16985
for <johnm@localhost>; Thu, 16 Mar 2000 01:23:19 -0800
From: Vivek_Magotra@palm.com
Received: from pop.enterprise.net
by localhost with POP3 (fetchmail-5.0.0)
for johnm@localhost (single-drop); Thu, 16 Mar 2000 01:23:19 -0800 (PST)
Received: from mail.acm.org (mail.acm.org [199.222.69.4])
by mail.enterprise.net (8.8.5/8.8.5) with ESMTP id JAA06113
for <jmarshall@enterprise.net>; Thu, 16 Mar 2000 09:14:10 GMT
Received: from seattle.3com.com (seattle.3com.com [129.213.128.97])
by mail.acm.org (8.9.3/8.9.3) with ESMTP id EAA17554
for <JMarshall@acm.org>; Thu, 16 Mar 2000 04:12:58 -0500
Received: from new-york.3com.com (new-york.3com.com [129.213.157.12])
by seattle.3com.com (8.8.8/8.8.8) with ESMTP id BAA15949
for <JMarshall@acm.org>; Thu, 16 Mar 2000 01:14:07 -0800 (PST)
Received: from hqpalmsmtp.palm.com (hqpalmsmtp.OPS.3Com.COM [139.87.49.252])
by new-york.3com.com (8.8.8/8.8.8) with SMTP id BAA29566
for <JMarshall@acm.org>; Thu, 16 Mar 2000 01:14:07 -0800 (PST)
Received: by hqpalmsmtp.palm.com(Lotus SMTP MTA v4.6.3 (778.2 1-4-1999)) id 882568A4.0032B913 ; Thu, 16 Mar 2000 01:14:01 -0800
X-Lotus-FromDomain: 3COM
To: JMarshall@acm.org
Message-ID: <882568A4.0032B7E3.00@hqpalmsmtp.palm.com>
Date: Thu, 16 Mar 2000 01:13:48 -0800
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
X-UIDL: 59727a30cf1836fc649f25f03f3b9f04
Hello PALM Programmers,
Lately I've been working with GLib libraries under PalmOS. On inspecting
the dynamic linking strategies, I've found the entire mechanism of dynamic
linking and access to global variables unnecessarily fragile and wish
to propose an alternative strategy.
First, a bit of background. I've been working on embedded processors
for 20 years. I've used microprocessors from Z80, 680x, 6800x, i80196,
x86, AMD a29k, and for the past 2 years, StrongARM. Our embedded development
uses a multi-threaded dynamically linked real-time OS developed specifically
for assembling multiple PIC reentrant code modules. Gosh, thats alot
of buzzwords. Anyway, over the years I have alot of experience
hacking GCC for our own purposes especially with regard to supporting
multiple code modules in a single address space.
>From what I've gleaned from the sources, the current GLib strategy
as of the prc-tools-2.0 code uses two different approaches for accessing
global data. A combination of the `-mowngp' and the `owngp' function
attribute controls code generation options regarding global data.
owngp Uses %a4 for the global base register, pointing to
the start of the data section. This activates
certain prologue and epilogue code which saves and
restores %a4 for methods deemed by the programmer to
be callbacks.
no-owngp Uses %a5 for the global base register, pointing to
the end of the data section. This mode disables
all `owngp' attributes for callback functions.
The idea is that in PalmOS there's typically only one application and
it normally calls one library. Applications use the global %a5 register
which (theoretically) should never change while the application is running
and points to the END of the application globals. GLib libraries use the
%a4 register which points to the START of the library's globals. Because
all GLib libraries must share a single %a4, each shared library entry
trampoline must load %a4. Unfortunately, a trampoline can't restore %a4
on return, since doing so would push registers on the stack and misalign
the formal parameters to the function.
Alternately, the called function could push %a4, load %a4 with its own
global base, and restore the register on return. Problem here is the
overhead of actually determining the global base register, %a4.
Current techniques appear to store a GLib's global base register within a
`LibRef' object within the caller. Thus, the stub procedure sets up
%a4 properly for the library being called from a static variable contained
in the CALLER. A function prologue within the called GLib has no access to
this static data in the caller and also has no global data itself. Thus,
it has no direct access to any storage to find its own global base register.
A caller can only be either an Application or a GLib; because, however,
C and C++ allow pointers to functions and methods, the callee can
be either a GLib stub or to the actual GLib via a pointer. The following
table summarizes the possibilities and expected result of each.
Caller Callee Description
------ ------ -----------
Application Application OK. Globals accessed via %a5
Application GLib stub OK. GLib trampoline loads %a4
Application GLib pointer Crash, especially with multiple GLibs.
GLib Application OK. Globals accessed via %a5
GLib GLib stub OK, GLib trampoline loads %a4
GLib GLib pointer Crash if different GLibs
In prc-tools-0.5, callback functions used macros called CALLBACK_PROLOGUE
and CALLBACK_EPILOGUE for function callbacks. These macros setup %a4
by subtracting the size of the data segment from %a5. This worked OK
for applications, but is now unnecessary with prc-tools-2.0 which has
the ability to access globals with a negative offset from %a5. Seems
to me, CALLBACK_PROLOGUE and CALLBACK_EPILOGUE only worked around
deficiencies in the linker which did not yet have the ability to create
negative relocations from %a5. In prc-tools-2.0, these macros seem to
have no purpose. In fact, they would surely be disastrous for GLib
callbacks, since they would subtract the size of the GLib data section
from the end of the Application data section. The result would be that
the GLib globals would overwrite the application globals. Pointers
to GLib functions cannot work in the current implementation.
One thing I can't yet figure out is why %a5 must point to the END of
the data segment? Does PalmOS access system globals with positive
offsets from %a5 and application globals with negative offsets? Or
is this just a silly convention?
The current system has problems handling callbacks to functions
within a GLib. One might say that this is just not supported, but
consider the case of C++ virtual functions. Objects constructed
within a GLib have vtables which directly reference the methods
within that GLib, not the shared library entry stubs. Calling C++
methods from outside the library that defines the object's class
is perilous and fragile. After an object is instantiated, if an application
invokes another GLib that reloads %a4, virtual functions in the first
GLib see invalid global data. In my opinion, this mechanism is
so fragile that the GLib mechanism can only be safely used for a single
library using %a4 and a single application using %a5.
In summary, the current GLib implementation several shortcomings:
1) Inability to properly handle virtual functions and callbacks.
2) Shared library entry stubs have added complexity since they must
handle opening the GLib and loading the %a4 register. Much
of this complexity can be handled once by global constructors.
3) The entry stubs are all compiled into a single .o file which
gets included by all applications which access the GLib.
Separating the stubs into separate .o files eliminates
alot of unused stubs from applications which access GLibs.
4) The entry table currently gets built by running `nm' on the
output of a `.a' file. Adding anything to the shared
library requires relinking all the applications that access
the library because all the entry points change positions.
Maintaining the ordering of entry points would achieve
upwards compatibility of GLib libraries.
How can we fix these problems? The dynamic linking strategy I have
currently developed for GCC on our embedded designs assigns an ordinal
to each shared library. In PalmOS we have a similar ordinal, the
CreatorID of each module. In our embedded designs, the ordinal is
a small number (0..255), so the our OS just maintains a vector of
data segments. In PalmOS, the ordinal is 32-bits so access is
a bit slower, but the FtrGet() PalmOS system call provides a clean
interface to store the GLib data segment for each library.
In object oriented C++, access to globals variables is typically less
frequent than in C because each method accesses state at offsets
from `this' rather than in static globals. My approach to static
variables in GCC allocates a global base register only if the current
method accesses static data. The function prologue allocates
a register and initializes the base register only if the function
needs it. In our embedded OS it's quite cheap to get this base
register since it just requires loading the N'th element of a vector
where `N' is a link-time constant. In PalmOS, using FtrGet() is a bit
more costly but still relatively cheap. However, we only need
to call FtrGet() when we enter a different library. Using one register
to store the CreatorID for the current global base, and another
register to store the global base allows us to only invoke FtrGet()
when switching libraries. Thus, the function prologue for any
function which accesses global data would execute the following
code:
push(%a4);
push(%a5);
if (%a4 != MyCreatorID) {
void *base;
FtrGet(MyCreatorID, FtrNumberGlobalBase, &base);
%a5=base;
}
The epilogue then restores %a4 and %a5.
pop(%a5);
pop(%a4);
This mechanism has many advantages:
1) The overhead for this mechanism is minimal because most
functions don't access global data and only invoke
FtrGet() when calling a different library.
2) This mechanism properly handles calls from any application to
any shared library.
3) This mechanism handles applications and GLib modules symetrically.
There isn't any requirement to compile GLib modules differently,
and the same .o files used for static linking can be used
for building a shared GLib module. The current mechanism
compiles GLib modules specially with the -mownfp flag.
4) Shared library entry stubs are simpler since they don't
worry about loading %a4.
5) This mechanism eliminates all the funky __attribute__ hacks
necessary to properly handle PalmOS callbacks!!! No more
CALLBACK_PROLOGUE; no more ENTRYPOINT() macros.
Because I really want to create GLib libraries using C++, I'll can
make these mods to the dynamic linking mechanisms, however they
fundamentally change the way %a4 gets handled in the environment.
I'd like to hear from all you developers out in PalmLand and get
your comments.
--
Jon Olson, Modular Mining Systems
3289 E. Hemisphere Loop
Tucson, AZ 85706
olson@mmsi.com
Phone: (520)746-9127
Fax: (520)889-5790
--------
The purpose of this email is to get input on how you'd like the time zone APIs
to work. There are three basic APIs: one for getting or setting the time zone
and daylight saving time, one for converting date/times between time zones and
one for converting back and forth between date/times and strings in various
formats.
Although I feel I have a handle on the first and second, I definitely need help
on the third. Anyone who currently uses routines for converting between
date/time (or second counts) and strings can really help here. What do you need?
Do you have code that does this which I can use to implement the new APIs? Are
the APIs I propose below a good fit for your needs?
Getting or setting time zone and daylight saving time:
Existing APIs:
Although these APIs are probably not used much, there are existing APIs for
getting and setting the time zone and daylight saving time. They're set with
PrefSetPreference and read with PrefGetPreference. The enum values passed are
prefMinutesWestOfGMT and prefDaylightSavings. The caller is expected to cast
these to UInt32 and DaylightSavingsTypes. Despite being called minutesWestOfGMT,
it is in fact measured east from GMT, and since the type is unsigned, it must
add 24 hours to represent time zones west of GMT, making it impossible to
distinguish GMT-12:00 and GMT+12:00 (opposite sides of the international date
line). DaylightSavingsTypes is an enum with values corresponding to various
rules for when daylight saving time is used. The specifics for these rules is
not given in the code or documentation. No support for manually setting DST is
provided.
Proposed APIs:
Add a new Int16 prefTimeZone preference, measured in minutes west of GMT, with a
range of + or - 24 hours. I call the new preference prefTimeZone, so that the
developer has to read the docs to find out how it's represented. The problem
with explicitly stating that it's minutes east or west of GMT is that this might
make the developer think that daylight saving time would affect it, when in
fact it does not. The docs would clearly state this. Should prefTimeZone measure
east or west from GMT? In a way, east makes more sense because that puts
GMT+1:00 at 60 rather than -60.
Add a new Boolean prefDaylightSavingTime preference.
Remove the old prefMinutesWestOfGMT and prefDaylightSavings preference. Keeping
both would be confusing to the developer.
Change the cnty resources for the countries to use the new time zone
representation. Keep the old DST representation in case we later want to support
automatic DST, and to aid in the initial setting for DST. The current list of
cities can be represented unambiguously with the existing representation, but
the confusing terminology makes it more difficult to manage than it should be.
Time Zone Conversion Utilities:
Existing APIs:
There are currently no routines for converting between time zones.
Proposed New APIs:
Now that we're providing a UI for the user to say where they are, and whether
daylight saving time is on, we can reasonably compute the current GMT from the
local time, or convert a GMT time to a corresponding local time. In order to
minimize the number of traps, we can use generic conversion routines that accept
the time zone and daylight saving time as arguments. All that is needed are
routines for converting from GMT to any other time zone, and vice versa:
void TimTimeZoneToGMT(DateTimePtr dateTimeP, Int16 timeZone,
Boolean daylightSavingTime);
void TimGMTToTimeZone(DateTimePtr dateTimeP, Int16 timeZone,
Boolean daylightSavingTime);
The developer can get the preferences for time zone and DST and pass them in to
the first routine to get GMT. Similarly, a time received with an email in GMT
can be converted to local time using the second routine. By making calls to the
first and second routine, it is possible to convert in either direction between
the local time zone and any other time zone.
Date/Time String Conversion Utilities:
Existing APIs:
There are currently routines for converting dates and times to strings in
various formats. None of these routines deals with GMT time; they all use local
time. There are no routines for converting ASCII representations of dates or
times to
seconds or date/times (parsing).
Proposed New APIs:
Now that we know the time zone, we can generate strings whose format includes
both a local time and time zone information. We can also generate strings in
formats which use GMT time. There are many formats for these conversions, used
for various things including internet email, SMS, etc. The exact specifications
for these formats are needed. Given the form of the existing routines for
converting dates and times to ASCII, I propose:
void DateTimeToAscii(DateTimePtr dateTimeP, Int16 timeZone,
Boolean daylightSavingTime, DateTimeFormatType format, Char *stringP);
typedef enum {
dtfRFC822 // the standard used for internet time
dtfRFC2425 // part of the vCard 3.0 standard
dtfRFC2426 // part of the vCard 3.0 standard
dtfRFC8601 // vCalendar 1.0 standard
} DateTimeFormatType;
DateTimeFormatType is an enum listing all standards we want to support. We use
an enum rather than a bit field because this is the way the existing conversion
routines work. All of these standards would take the given time zone and
daylight saving time into account. The date/time given would be assumed to be in
the given time zone/DST. If the format is GMT based, the routine would convert
the date/time from the given zone to GMT. If the format is based on specifying a
time and a zone, it'd specify the given time and zone. For typical use, the
date/time would be given in local time and the time zone would be the local time
zone. When a format supports multiple variants, any one may be produced by this
routine. This is acceptable because any program which conforms to the format
standard must be able to parse all variants. Note that the routine doesn't have
a strLen argument. This corresponds to the way DateToAscii and TimeToAscii work
now; the caller is expected to allocate a buffer of a specific size that is b
ig enough to fit all possible output strings. We'd define constants for the max
length of strings for each format.
We also need routines to convert ASCII representations to date/times. No doubt,
there are applications which need this service, and it makes sense to move this
code into the operating system.
Err AsciiToDateTime(const Char *stringP, DateTimeFormatType format,
DateTimePtr dateTimeP, Int16 *timeZoneP, Boolean *daylightSavingTimeP);
The same format enum is used to specify the format of the string. Note that a
time zone and DST value are returned with the date/time. This allows ASCII
representations which include a time and zone information to be completely
described (allowing a senders local time to be displayed with a received email
message). For formats which use GMT, the time zone returned would always be GMT,
and the DST returned would always be false. When a format supports multiple
variants, all must be handled by this routine.
I don't think we need functions for extracting information about the format of a
date/time string. If you need to parse a string and you don't know which format
it's in, you can always try each format and see which give errors and which do
not. If multiple succeed, and the results differ, then you have a problem.
Some ASCII representations separate the time zone specification from the
date/times. To support these formats, we would need separate conversion
routines. However, as far as I can tell, the only standard with this problem is
iCalendar (the new vCalendar spec). Unless we have specific plans to use this
complex standard, I don't think we should implement these separate conversion
routines, but if we do, they'd look something like this:
typedef enum {
tzfPalmShort // as used in Date & Time panel for trigger text
tzfPalmLong // as used in Time Zone dialog for list of time
zones
tzfRFC822 // the standard used for internet time
tzfRFC2425 // part of the vCard 3.0 standard
tzfRFC2426 // part of the vCard 3.0 standard
tzfRFC2445 // iCalendar standard
tzfRFC8601 // vCalendar 1.0 standard
} TimeZoneFormatType;
Err AsciiToTimeZone(const Char *stringP, TimeZoneFormatType format,
Int16 *timeZoneP, Boolean *daylightSavingTimeP);
void TimeZoneToAscii(Int16 timeZone, Boolean daylightSavingTime,
TimeZoneFormatType format, Char *stringP);
This last function could also be used to get the short trigger text or the
longer text we use in the list of time zones.
Summary:
The APIs I propose look fairly clean and use only a small number of traps.
However, they may be missing critical functionality. If they are, I need to know
about it. Do we need to support parsing formats in which the time zone is
separate from the date/time? Thanks for reading this far!
--
Peter Epstein
More information about the Gcc
mailing list