FreshLib user guide # Chapter 7 FreshLib user guide ----------------------------------- ## Work with dynamic memory. ### Use with heap manager FreshLib provides uniform and OS independent way to work with dynamic memory blocks. The API is simple - there are only 3 procedures: `proc GetMem, .size` - allocates new block with size at least `[.size]` and returns a pointer to this block in EAX. In case of error the procedure returns EAX=0 and CF=1. `proc FreeMem, .ptr` - frees already allocated by GetMem block of memory. `proc ResizeMem, .ptr, .newsize` - changes the size of some memory block, allocated by GetMem. The implementation of these procedures differs on the different OSes. In Win32, these procedures are implemented using Win32 heap functions. In Linux, using libc functions. There is an another approach to the memory management - FreshLib has its own implementation of a heap manager, that is OS independent and can be used in multiply operating systems. What heap management code will be used is defined through the `HeapManager` preprocessor symbol to "ASM". If `HeapManager` is set to `OS`, the OS specific memory management will be used. If `HeapManager` is set to `ASM` the FreshLib heap manager will be compiled. The FreshLib heap manager must always be used for OSes that has no their own method of heap management. For example KolibriOS port of the library uses this heap manager by default. The heap manager procedures in FreshLib are thread safe. ### Use without heap manager The heap management is not mandatory and can be disabled at all. In this case the application will be able to allocate the whole needed memory as one big chunk. There is only one procedure about this approach: `proc SpaceAllocate, .size` - allocates memory with the specified size. In this case, the start address of the allocated memory is placed in the global variable `_MemoryFreeSpace` and will not be changed during the program execution. Only the size of the available block of RAM can be changed by the above procedure. Unfortunately, big amount of FreshLib functionality depends on the heap managemer. This way, if you are using this memory management approach, this functionality can not be used in the user program. That is why, generally, this approach is not recommended for wide use. Although, there are some situations (for example simple program, that needs big, contiguous block of memory) where such simple approach can give you big performance gain without the mentioned disadvantages. ----------------------------------- ## Work with dynamic strings The strings in FreshLib are dynamic - their length is changed in order to hold the whole information user put into the string. Because of this, the strings are represented not by their address, but by handle. However, most of the procedures in the library can operate with memory pointers as well. The strings in FreshLib are always UTF-8 encoded. ### Internal string format The library distinguish the handles from the pointers, because of the two most significant bits (30th and 31st) of the handle are always set to 1. (this approach can rarely create some problem with systems where such high addresses are used, but usually it is OK). In order to create new string, the user should use the procedures `StrNew` or `StrDup`. The first creates new empty string. The second, creates new string with content copied from some existing string. Some of the strings processing procedures also can create new string and return it as a result value. ### Short list of the string manipulating procedures *Here is a table with most used string functions with short description:* ;table *Procedure* *Arguments* *Operation* *Notes* ;------------------------------------------------------------------------------------------------- `StrNew` ..... Creates new string ..... ;------------------------------------------------------------------------------------------------- `StrDup` .hString Creates new string from existing one. ..... ;-------------------------------------------------------------------------------------------------- `StrPtr` .hString Returns a pointer to the current string memory. ..... ;-------------------------------------------------------------------------------------------------- `StrDel` .hString Deletes some string. Preserves all registers and flags. ;-------------------------------------------------------------------------------------------------- `StrLen` .hString Returns the length of the string in bytes. This procedure is very fast for direct pointers and instant for FreshLib strings. ..... ;-------------------------------------------------------------------------------------------------- `StrCopy` .dest, .source Copies the content of the `[.source]` string to the `[.dest]` string. ..... ;-------------------------------------------------------------------------------------------------- `StrCat` .dest, .source Concatenate the content of `[.source]` at the end of `[.dest]` string. ..... ;-------------------------------------------------------------------------------------------------- `StrCopyPart` .dest, .source, .pos, .len Copies a part of `[.source]` string - from byte `[.pos]` with length `[.len]` to the `[.dest]` string. ..... ;-------------------------------------------------------------------------------------------------- `StrExtract` .string, .pos, .len Extracts a part of the string `[.string]` (from byte `[.pos]` and with length `[.len]`) and returns it as a new created string. ..... ;-------------------------------------------------------------------------------------------------- `StrSplit` .hString, .pos Splits the string by two parts. The left part remains in `[.hString]` and the right part remains in new created string, returned by the procedure. ..... ;-------------------------------------------------------------------------------------------------- `StrTrim` .hString, .pos Trims the string at position `[.pos]` The remaining of the string is lost. ..... ;-------------------------------------------------------------------------------------------------- `StrInsert` .dest, .source, .pos Inserts the string `[.source]` in the string `[.dest]` at byte `[.pos]`. ..... ;-------------------------------------------------------------------------------------------------- `NumToStr` .num, .flags Converts the 32bit number `[.num]` into a new string, depending on the flags `[.flags]`. Flags: ((ntsSigned | ntsUnsigned)+(ntsFixedWidth)) + (ntsBin | ntsQuad | ntsOct | ntsDec | ntsHex) ;-------------------------------------------------------------------------------------------------- `StrToNumEx` .hString Converts a string containing FASM formated number into a number. ..... ;-------------------------------------------------------------------------------------------------- `StrCharCat` .hString, .char Concatenates to the string `[.hString]` up to 4 bytes of ascii (UTF-8) characters from `[.char]`. ..... ;-------------------------------------------------------------------------------------------------- `StrCharInsert` .hString, .char, .pos Inserts up to 4 characters from `[.char]` into the string `[.hString]` at the position `[.pos]`. ..... ;-------------------------------------------------------------------------------------------------- `StrClipSpacesR` .hString Removes all trailing spaces from the string `[.hString]`. ..... ;-------------------------------------------------------------------------------------------------- `StrClipSpacesL` .hString Removes all leading spaces from the string `[.hString]`. ..... ;-------------------------------------------------------------------------------------------------- `StrCleanDupSpaces` .hString Removes all duplicated spaces from `[.hString]` and leaves only one space on this place. ..... ;-------------------------------------------------------------------------------------------------- `StrHash` .hString Computes and returns FNV-1b hash of the string `[.hString]`. ..... ;-------------------------------------------------------------------------------------------------- `DataHash` .ptrData, .len Computes the FNV-1b hash of the memory area from `[.ptrData]` with length `[.len]`. ..... ;-------------------------------------------------------------------------------------------------- `StrURLEncode` .hString Encodes the string in `[.hString]` in URL encoding. ..... ;-------------------------------------------------------------------------------------------------- `StrURLDecode` .hString Decodes URL encoded string in place. ..... ;-------------------------------------------------------------------------------------------------- `StrLenUtf8` .hString Computes the length in chars of the UTF-8 encoded string in `[.hString]` This operation is relatively slow because the string must be scanned byte by byte. ..... ;-------------------------------------------------------------------------------------------------- `StrOffsUtf8` .hString, .pos Returns the byte offset of the character position `[.pos]` in the UTF-8 encoded string `[.hString]`. ..... ;-------------------------------------------------------------------------------------------------- `StrExpandTabs` .hString, .tabstop Converts all tabs in the string `[.hString]` into a spaces using `[.tabstop]` as a tab distance. ..... ;-------------------------------------------------------------------------------------------------- `StrIP2Num` .hString Converts the string representation of IP address into a 32bit number. ..... ;-------------------------------------------------------------------------------------------------- `IP2Str` .ip Converts numeric representation of an IP number `[.ip]` into a string. ..... ;-------------------------------------------------------------------------------------------------- `StrEncodeHTML` .hString Encodes the string `[.hString]` in HTML encoding. ..... ;-------------------------------------------------------------------------------------------------- `StrDecodeHTML` .hString Decodes the HTML encoded string `[.hString]`. ..... ;-------------------------------------------------------------------------------------------------- `DateTimeToStr` .pDateTime, .format Converts TDateTime structure into a string, depending on `[.format]` flags. ..... ;-------------------------------------------------------------------------------------------------- `StrMD5` .hString Computes MD5 sum of the string `[.hString]`. ..... ;-------------------------------------------------------------------------------------------------- `StrSplitList` .hString, .Separator, .fAutoEmpty Creates dynamic array of dword, filled with the parts of the string `[.hString]`, splitted on the character `[.Separator]`. The returned array must be freed after use with: `stdcall ListFree, [RetPtr], StrDel`. ..... ;-------------------------------------------------------------------------------------------------- `AnsiToUtf8` .SrcEncoding, .hString Converts an ANSI, `[.SrcEncoding]` encoded string into UTF-8 encoding. WIN1251, CP866, KOI8R and KOI8U are currently supported, but FreshLib has tool that creates encoding tables from UNICODE mapping tables from [http://www.unicode.org/Public/MAPPINGS/] ;-------------------------------------------------------------------------------------------------- `Utf8ToAnsi` .hString, .DstEncoding Converts UTF-8 encoded string into an ANSI encoded with encoding `[.DstEncoding]` ..... ;end table ### String procedures example Here is some simple example demonstrating the use of several string functions: ;begin iglobal MyConst1 db 'Mickey ', 0 MyConst2 db 'Mouse ', 0 endg stdcall StrNew mov ebx, eax stdcall StrDup, MyConst1 stdcall StrCat, eax, MyConst2 ; note, that StrCat does not change any registers. stdcall StrCat, ebx, eax stdcall StrDel, eax ; StrDel also preserves the flags. ;end ----------------------------------- ## Using dynamic libraries other than the standard. FreshLib creates all the import tables needed for the program automatically. Usually, the functions imported from external DLLs are not supposed to be called from the user code, because such calls are not portable. But sometimes, the programmer needs some functionality that is accessible only through an external library. There are two cases: 1. The internal library is available only for one particular OS. In this case, the result program will be not portable and nothing can be done. 2. There are versions of the same library for all target platforms. This is the best variant that will allow to keep the created program portable. But in this variant, there is a catch. It requires the calling conventions of the different OS versions of the library to be the same. In order to use external DLLs, the one will need a file with definitions of the imported functions placed somewhere in the project directory. This file should contain a list with the functions exported from the DLL using the FreshLib macro 'import_proto'. [#import_proto] macro is defined for Linux and Windows, following way: ;begin macro import_proto library, [iname, arg, ename] ;end Where: `library` is the name of the DLL; `iname` is the name of the function as it will appears in the FreshLib program. `arg` is an optional list with the function's arguments. They will be used for function call argument hinting `ename` is a string with the name of the function as exported from the DLL. The only difference between the Windows and Linux version is the value of *library* field. In Windows it is the name of the DLL without the extension. In Linux it is a string with the full name of the shared library. Here are two examples of this macro use for sqlite3 library for Windows and Linux (here are only the first 4 function definitions): In *Windows:* ;begin import_proto sqlite3, \ sqliteAggregateContext, <.context, .nbytes>, 'sqlite3_aggregate_context', \ sqliteAggregateCount_OLD, , 'sqlite3_aggregate_count', \ sqliteAutoExtension, <.ptrCallback>, 'sqlite3_auto_extension', \ sqliteBackupFinish, <.ptrBackup>, 'sqlite3_backup_finish', \ ;end In *Linux:* ;begin import_proto 'libsqlite3.so.0', \ sqliteAggregateContext, <.context, .nbytes>, 'sqlite3_aggregate_context', \ sqliteAggregateCount_OLD, , 'sqlite3_aggregate_count', \ sqliteAutoExtension, <.ptrCallback>, 'sqlite3_auto_extension', \ sqliteBackupFinish, <.ptrBackup>, 'sqlite3_backup_finish', \ ;end In the argument definitions, there are two special values: `` means that the arguments are unknown and `` means the function does not has any arguments. All arguments must start with '.', i.e. to be defined as local labels. In the above example, the function `sqlite3_aggregate_context` has two arguments `.context` and `.nbytes` and is imported with the name `sqliteAggregateContext`. If several definition files for different OSes are created, they have to be placed in separate directories, named after the OS they are targeted for. For the above example the first file will be placed in "myproject/Win32/sqlite.inc" and the second in "myproject/Linux/sqlite.inc". If only one OS definition file is created, it may be placed anywhere. The use of these definition files is with the macro `uses`, placed somewhere in your code that uses this library. The macro is defined following way: [#uses] ;begin macro uses [libname] ;end Where `libname` is the name of the library in format: `LIBRARY_NAME[:definition_filename]`. The filename of the definition file is optional and if omitted is created automatically as '%lib%/imports/%TargetOS%/LIBRARY_NAME.inc If several definition filenames are created for different OSes, the uses macro must contain %TargetOS% environment variable. For example: ;begin uses sqlite3:'myproject/%TargetOS%/sqlite3.inc' ;end Defined this way functions can be used in the program from everywhere and later, when @AllImportSection or @AllImportEmbeded is used, FreshLib will automatically create the import table of the executable. Notice, that only the functions that are used will be imported. This way, it is always safe to use `uses` macro. It will never create over-bloated import table.