This EPICS driver gives access to the resources of the "Tosca" FPGA framework on IOxOS boards like the IFC1210, IFC1211 or IFC1410. It interfaces the "tosca" Linux kernel driver written by Denx. It is incompatible with the alternative kernel drivers "pev" and "althea" by IOxOS for the same boards!
The software is divided into parts: A non-EPICS library provides a C API to the Tosca resources. Any program can use this library without the need for EPICS.
But several EPICS interfaces also use this API: IOC shell functions mainly for debugging, a devLibVME interface for EPICS drivers using the standard VME interface, and a regDev interface to access all Tosca resources (not only VME) in a generic way and to use DMA where possible.
It is assumed the PSI build and load system for EPICS modules is used. If a different system is used, a different Makefile than the provided GNUmakefile must be written and loading the resulting library may be done differently than described here.
The API and EPICS driver have been developed for and tested with the IOxOS
IFC1210, IFC1211 and IFC1410 boards but may also work with other IOxOS
products using the Tosca framework and kernel driver.
It is required to use Tosca Central and Pon FPGA firmware that is
compatible with the "tosca" kernel driver.
In particular, the kernel driver must show the Tosca device in
/sys/bus/pci/drivers/tosca/.
It is possible to handle multiple Tosca devices in the same system. One example is the IFC1211 where the VME interface is on device 0 while the USER FPGA interface is on device 1. Also connecting multiple IFC boards with PCIe and running the second as a slave of the first should be possible but this has never been tested.
This software requires the following external software modules:
- Tosca kernel driver
- symbolname
- memDisplay
- sysfs
- EPICS base release 3.14.12 or higher, when built with EPICS support.
- keypress, when built with EPICS iocsh support.
- regDev, when built with EPICS register device support.
- i2cDev, when build with pev compatibility.
- pev header files, when build with pev compatibility.
This software is published under the terms of the GNU General Public License version 3 or any later version. See [https://www.gnu.org/licenses/].
The C API provides access to the following Tosca resources:
- Memory maps of several Tosca resources.
- Tosca USER1 and USER2 blocks in the Central FPGA
- Tosca SMEM1 and SMEM2 shared memory
- Tosca TCSR and TIO configuration space registers
- Tosca SRAM through PON FPGA and ELB
- VME master in A16, A24, A32, A64 and CRCSR address space (Tosca currently does not support A64.)
- VME SLAVE in with mapping to memory, USER, SMEM (Tosca currently supports slave windows only on A32 address space.)
- Register access functions which read or
write registers on tosca resources as 32 bit words and take care of byte
swapping and atomic access where necessary.
- TCSR and TIO Tosca configuration registers
- PON FPGA registers through ELB
- Generic for TCSR, TIO, USER, SMEM ...
- SMON Virtex FPGA system monitor registers through TCSR
- DMA transfers.
- VME A32 <-> RAM, SMEM, USER, VME A32
- Single 32 bit transfers
- BLT
- MBLT
- 2eVME
- 2eSST (160, 267, 320)
- USER, SMEM <-> RAM
- USER <-> SMEM
- VME A32 <-> RAM, SMEM, USER, VME A32
- Interrupt handler connection for several interrupt sources.
- USER1 and USER2 (16 interrupt lines each)
- VME (7 levels with 256 vectors)
- VME errors (sys fail, ac fail, bus error)
- Interrupt generation on VME.
Currently implemented Tosca resources depending on the board type:
- IFC1210: SMEM1, USER1, VME, TCSR, TIO, SRAM
- IFC1211 VME, SMEM1, TCSR, TIO, SRAM, 1:USER1, 1:USER2, 1:SMEM1, 1:SMEM2, 1:TCSR, 1:TIO
- IFC1410: TCSR, TIO, USER1, USER2, SMEM1, SMEM2, SRAM
I²C devices on the board can be handled by the standard Linux I²C support but see also the simple I²C API described later.
For debugging purposes, the API functions are also available from the EPICS IOC shell.
To build your own non-EPICS C/C++ programs with the Tosca API, some
compiler and linker flags are required in order to find the API library
and header files.
A program can use either the static or the shared API library.
For convenience a single header file toscaApi.h can be included which
in turn includes all other required header files.
Preprocessor flags:
-I $PATH_TO_IFC_ROOT_DIRECTORY/usr/local/includeShared library linker flags:
-L $PATH_TO_IFC_ROOT_DIRECTORY/usr/local/lib/ -l toscaApi -Wl,-rpath,/usr/local/libStatic library linker flags:
$PATH_TO_IFC_ROOT_DIRECTORY/usr/local/lib/libtoscaAPI.a -lpthread -lrt -ldlHeader file:
#include "toscaApi.h"Hint for printing Tosca addresses: The Tosca API uses 64 bit values for
addresses. The underlying C data type depends on the CPU architecture
(32 or 64 bit). This makes it a bit inconvenient to find the correct
printf() converter ("%x" or "%lx" or "%llx").
Best use the converter macros from inttypes.h, e.g "%"PRIx64.
unsigned int toscaNumDevices(void);Returns the number of available Tosca devices.
The devices are numbered 0 ... toscaNumDevices() - 1 in the order they
appear in a directory listing of /sys/bus/pci/drivers/tosca/.
unsigned int toscaDeviceType(unsigned int device);Returns the PCI device type of the given Tosca device. One finds the the following:
- IFC1210: toscaDeviceType(0) = 0x1210
- IFC1211: toscaDeviceType(0) = 0x1211, toscaDeviceType(1) = 0x1001
- IFC1410: toscaDeviceType(0) = 0x1001
unsigned int toscaListDevices();Prints information about the found Tosca devices to stdout.
volatile void* toscaMap(unsigned int addrspace, unit64_t address, size_t size, uint64_t res_address);
volatile void* toscaMapMaster(unsigned int addrspace, unit64_t address, size_t size);This function creates a new or re-uses an existing map of a master or
slave window to a Tosca resource.
It will try to map at least size bytes starting at address in the
selected address space addrspace.
If a matching map already exists, it will be re-used. Thus the function
can safely be called multiple times without wasting resources.
But to keep this simple there is no unmap.
The maps and associated kernel and Tosca resources are released
automatically when the program terminates.
For addrspace use one of VME_A16, VME_A24, VME_A32,
VME_A64, VME_CRCSR, TOSCA_USER1, TOSCA_USER2,
TOSCA_SMEM1, TOSCA_SMEM2, TOSCA_CSR, TOSCA_IO, or
TOSCA_SRAM.
For VME, addrspace can be combined (bitwise or) with
|VME_SUPER for "supervisory" access and/or
|VME_PROG for "program" access.
In case more than one Tosca device is available, use
(addrspace)|(device<<16).
If addrspace is invalid, the behavior is undefined
(may fail or giveyou a map you did not expect).
Not all address spaces are supported by all Tosca implementations.
At the moment none implements VME_A64.
The IFC1210 does not implement TOSCA_USER2 or TOSCA_SMEM2.
The IFC1211 implements VME only on device 0 and USER only on device 1.
Also TOSCA_SMEM2 on device 0 is not accessible.
The IFC1410 does not implement VME.
The function will fail and return NULL if you try an unsupported map.
In EPICS memory maps can be accessed using the regDev interface.
Master maps give access to Tosca resources like VME, USER or SMEM to the
program. They are mapped into user space and are accessible through the
returned pointer much like local memory.
Due to the granularity of Tosca mapping windows, the map actually created
may be larger than requested in both directions.
It may start at a lower address than address and may cover higher
addresses than address+size.
It is not necessary to call the function with aligned address or
size to the Tosca window granularity.
In any case the returned pointer corresponds to the requested address,
even if the actual mapping window starts at an earlier address.
Do not access any location before the returned pointer or after and
including the returned pointer + size.
If the request cannot be fulfilled, the function returns NULL and
sets errno.
Master maps no not use the res_address argument.
Pass 0 or use the macro toscaMapMaster() which does exactly this.
Be aware that Tosca resources may use byte orders different from the CPU and data may only be meaningful with correct data width and alignment. In particular TCSR and TIO registers are 4 byte aligned little endian while VME uses big endian and allows 1, 2, 4, or 8 byte access. Use htole32() or le32toh() (from endian.h) where necessary. The register access functions described below take care of this.
Tosca supports SLAVE maps that expose program memory or another Tosca resource like USER or SMEM on the VME bus. This makes local resources accessible by other VME cards. Of course VME SLAVE maps are only possible on Tosca devices that support VME.
To use SLAVE maps set addrspace to a combination (bitwise or) of
VME_SLAVE, one of the VME address spaces VME_A* and, if not
mapping to memory, another Tosca resource like TOSCA_USER1.
If combined with VME_SWAP the data will be swapped when accessed from
VME according to the access data width.
This is probably useful for mapping USER to VME because the registers on
USER are likely to be in little endian byte order while VME uses big
endian byte order according to the VME standard.
The byte order of SMEM data depends on the application.
The current Tosca implementation supports VME SLAVE maps only on
VME_A32 address space and can only map memory or the USER and SMEM
resources.
For convenience, VME_A32 is implicit if no VME address space is given
with VME_SLAVE.
Even though Tosca supports multiple VME SLAVE maps, all must lie in the same 512 MB of the VME address range. The kernel driver currently only supports the first 512 MB. All VME SLAVE maps have a granularity of 1 MB and different SLAVE maps cannot overlap. Thus it is not possible to allocate less than one MB in the A32 address space or to map two different resources to the same MB.
If mapping a VME SLAVE to USER or SMEM, the address on that resource is
needed in addition to the VME address.
Set res_address to the USER or SMEM address which should be mapped
to address on VME.
Both must have the same offset from 1 MB alignment, which means that the
lower 20 bits must be identical in address and res_address
(for example 0).
In this case the returned value is NULL even on success, because nothing
is mapped to memory and thus there is no pointer to return.
Use errno to test for errors, that means set errno to 0 before
calling toscaMap() and check for a non-zero value immediately afterwards.
To map program memory to VME set addrspace to VME_SLAVE|VME_A32
and pass 0 for res_address.
The current IFC boards use big endian CPUs, thus one should not set
VME_SWAP.
This would be different if a Tosca device is accessed by a little endian
CPU.
The function returns a pointer to memory visible on the VME bus. A maximum of 4 MB (after alignment to full MBs) can be mapped this way because the Linux kernel does not allow to allocate more linear address space.
If mapping fails, toscaMap() returns NULL and sets errno to one of
the following values:
ENODEVTosca device not foundEFAULTaddress or size out of rangeEINVALinvalid address spaceEADDRINUSEoverlap with existing VME SLAVE windowEACCESno permission to read and write Tosca deviceENOMEM,EMFILE,ENFILE,EAGAINinsufficient system resources
toscaMapInfo_t toscaMapForEach(int(*func)(toscaMapInfo_t info, void *usr), void *usr);
toscaMapInfo_t toscaMapFind(const volatile void* ptr);
toscaMapAddr_t toscaMapLookupAddr(const volatile void* ptr);The toscaMapForEach() function calls a user specified callback function for each installed map until the callback returns a value other than 0. The result describes the map for which the callback returned the non 0 value.
The toscaMapInfo_t type is a structure with the following fields
(in unspecified order):
unsigned int addrspace;
uint64_t baseaddress;
size_t size;
volatile void* baseptr;The toscaMapFind() function returns the description of the map in which the
passed ptr lies. (It uses toscaMapForEach().)
The toscaMapLookupAddr() function translates the passed ptr to the
following structure describing to which Tosca resource and address the
pointer refers. (It uses toscaMapFind().)
The toscaMapAddr_t type is a structure with the following fields
(in unspecified order):
unsigned int addrspace;
uint64_t address;All three functions set addrspace to 0 if no match is found.
These functions are not necessarily quick because they have to check each
installed map.
Debugging: The global variable toscaMapDebug can be set to enable
debug output, either to stderr or to toscaMapDebugFile if that global
FILE* variable is set.
unsigned int toscaRead(unsigned int addrspace, unsigned int address);
unsigned int toscaWrite(unsigned int addrspace, unsigned int address, unsigned int value);
unsigned int toscaSet(unsigned int addrspace, unsigned int address, unsigned int bitsToSet);
unsigned int toscaClear(unsigned int addrspace, unsigned int address, unsigned int bitsToClear);
unsigned int toscaCsrRead(unsigned int address);
unsigned int toscaCsrWrite(unsigned int address, unsigned int value);
unsigned int toscaCsrSet(unsigned int address, unsigned int bitsToSet);
unsigned int toscaCsrClear(unsigned int address, unsigned int bitsToClear);
unsigned int toscaIoRead(unsigned int address);
unsigned int toscaIoWrite(unsigned int address, unsigned int value);
unsigned int toscaIoSet(unsigned int address, unsigned int bitsToSet);
unsigned int toscaIoClear(unsigned int address, unsigned int bitsToClear);
unsigned int toscaSmonRead(unsigned int address);
unsigned int toscaSmonWrite(unsigned int address, unsigned int value);
unsigned int toscaSmonWriteMasked(unsigned int address, unsigned int mask, unsigned int value);
unsigned int toscaSmonSet(unsigned int address, unsigned int bitsToSet);
unsigned int toscaSmonClear(unsigned int address, unsigned int bitsToClear);
unsigned int toscaPonRead(unsigned int address);
unsigned int toscaPonWrite(unsigned int address, unsigned int value);
unsigned int toscaPonWriteMasked(unsigned int address, unsigned int mask, unsigned int value);
unsigned int toscaPonSet(unsigned int address, unsigned int bitsToSet);
unsigned int toscaPonClear(unsigned int address, unsigned int bitsToClear);
unsigned int toscaSbcRead(unsigned int fmc_slot, unsigned int reg);
unsigned int toscaSbcWrite(unsigned int fmc_slot, unsigned int reg, unsigned int value);
unsigned int toscaSbcWriteMasked(unsigned int fmc_slot, unsigned int reg, unsigned int mask, unsigned int value);
unsigned int toscaSbcSet(unsigned int fmc_slot, unsigned int reg, unsigned int bitsToSet);
unsigned int toscaSbcClear(unsigned int fmc_slot, unsigned int reg, unsigned int bitsToClear);
toscaMapVmeErr_t toscaGetVmeErr(unsigned int device);This is a set of convenience functions to access registers on Tosca resources. Internally they use the memory maps described above but simplify the usage. They assume 32 bit registers and little endian byte order on the Tosca resources. They take care of swapping from and to host byte order if necessary and assure atomicy of read-modify-write actions (for concurrent access from different threads in the same process, atomicy across process could only be provided by the kernel).
In case more than one Tosca device is available, combine (bitwise or)
the first argument with device<<16. Not all resources are available on
all devices.
The *Set(), *Clear() and *WriteMasked() functions atomically set the given bits to 1 or 0 respectively and leave the other bits untouched. The *Write(),*WriteMasked(), *Set(), and *Clear() functions return the new register value, which may be different from the value written due to read-only bits in the register.
In case of errors these functions return (unsigned int)-1 and set
errno. On success they set errno to 0.
In EPICS the registers can be accessed using the regDev interface.
Debugging: The global variable toscaRegDebug can be set to enable
debug output, either to stderr or to toscaRegDebugFile if that global
FILE* variable is set.
The functions toscaRead(), toscaWrite(), toscaSet() and toscaClear()
can access all toscaResources that can be mapped.
That is TOSCA_CSR, TOSCA_IO, TOSCA_USER1, TOSCA_USER2,
TOSCA_SMEM1, TOSCA_SMEM2, SRAM and and VME map.
(However it makes not much sense to use VME here because these functions
assume little endian byte order while VME uses big endian.)
The passed address should be a multiple of 4, at least for the CSR, IO
and USER address spaces.
The specific toscaCsr*() and toscaIo*() functions are simply shortcuts
for using TOSCA_CSR or TOSCA_IO as addrspace in the generic
functions and exist mainly for backward compatibility.
On the IFC1210 the register set on CSR and IO space are different while on IFC1211 and IFC1410 they contain the same registers.
Access to addresses up to 0x1000 should be well considered because it may interface with the internal functions of the device and may cause system crash or other unexpected behaviour. You have been warned.
The toscaPon* functions access PON FPGA registers over the processor local
bus. These cannot be accessed with the generic access functions.
The address range of the PON registers is limited to 0x00 to
0x24 plus 0x40.
The register map is as follows:
| address | register name |
|---|---|
| 0x00 | vendor |
| 0x04 | static_options |
| 0x08 | vmectl |
| 0x0c | mezzanine |
| 0x10 | general |
| 0x14 | pciectl |
| 0x18 | user |
| 0x1c | signature |
| 0x20 | cfgctl |
| 0x24 | cfgdata |
| 0x40 | bmrctl |
For details see the IFC hardware documentation.
The toscaSmon*() functions access the Virtex (Central) FPGA system monitor using the register pair TCSR:0x40/0x44. These should not be accessed with the generic access functions or directly using the CSR memory map because two registers are involved and atomicy cannot be ensured when not using the toscaSmon*() functions.
The address range of the Smon registers is limited to 0x00 to
0x7c and only registers above 0x40 are writable.
For more information refer to the Virtex documentation.
The toscaSbc*() functions access registers on FMC 1 or 2 using the serial bus controller register pairs TCSR:0x120c/0x1210 and TCSR:0x130c/0x1310. These should not be accessed with the generic access functions or directly using the CSR memory map because two registers are involved and atomicy cannot be ensured when not using the toscaSbc*()_ functions.
The address is a 30 bit combination of component id and register number on that
component. The register part is usually the lower 8 bits.
Details depend on the FMC plugged in.
For the ADC/DAC 311x family the components addresses are as follows:
| address | ADC3110/1 | ACD3112 | DAC311x |
|---|---|---|---|
| 0x01000000 | ads01 | ads01 | ads |
| 0x01010000 | ads23 | ads23 | ad9783 |
| 0x01020000 | ads45 | xra01 | |
| 0x01030000 | ads67 | xra02 | |
| 0x02000000 | lmk | lmk | |
| 0x02010000 | sy | ||
| 0x03000000 | dac | dacadj0 | |
| 0x03010000 | xratrig | dacadj1 | |
| 0x03020000 | daccmp0 |
For details refer to the hardware documentation.
The function toscaGetVmeErr() returns and re-arms the VME Error register pair TCSR:0x418/0x41c. These registers store address and status of the first VME error after it had been re-armed.
The function returns the following structure:
typedef struct {
uint64_t address; /* Lowest two bits are always 0. */
union {
unsigned int status:32;
struct {
unsigned int err:1; /* Error has happened since last readout. */
unsigned int over:1; /* [multiple errors, not implemented] */
unsigned int write:1; /* Error was on write access. */
unsigned int timeout:1; /* Error was a bus timeout */
unsigned int source:2; /* 0=PCIe 2=IDMA 3=USER */
unsigned int id:17; /* [What is this?] */
unsigned int length:5; /* [In words, for block transfer modes ?] */
unsigned int mode:4; /* 0=CRCSR 1=A16 2=A24 3=A32 4=BLT 5=MBLT
6=2eVME 8=2eSST160 9=2eSST267 10=2eSST320 15=IACK */
};
};
} toscaMapVmeErr_t;Pev compatibility note: When converting from pev functions
pev_csr_rd() and pev_csr_wr() to Tosca functions, be aware that the pev
functions could access both, the TIO and TCSR address space and used the
highest address bit (0x80000000) to select TCSR address space while in
Tosca different maps are used.
Thus if you had addresses x > 0x8000000 change to toscaCsr*()
functions with address x - 0x80000000 while with addresses
x < 0x8000000 change to toscaIo*() functions using address x.
For example, for reading user blocks TCSR registers, instead of using
pev_csr_rd(0x80001000+offset) one should use
toscaCsrRead(0x1000+offset).
typedef void (*toscaDmaCallback)(void* usr, int status);
int toscaDmaTransfer(unsigned int source, uint64_t source_addr,
unsigned int dest, uint64_t dest_addr,
size_t size, unsigned int swap, int timeout,
toscaDmaCallback callback, void* user);
int toscaDmaWrite(void* source_addr,
unsigned int dest, uint64_t dest_addr,
size_t size, unsigned int swap, int timeout,
toscaDmaCallback callback, void* user);
int toscaDmaRead(unsigned int source, uint64_t source_addr,
void* dest_addr,
size_t size, unsigned int swap, int timeout,
toscaDmaCallback callback, void* user);These functions perform a DMA transfer of size bytes from address
space source address source_addr to address space dest address
dest_addr.
All addresses and size must be multiples of 4.
The choices for source and dest are 0 (memory),
TOSCA_USER1, TOSCA_USER2, TOSCA_SMEM2, TOSCA_SMEM2, or one
of the VME block transfer modes VME_SCT (A32 single 32 bit transfers),
VME_BLT, VME_MBLT, VME_2eVME, VME_2eSST160,
VME_2eSST267, or VME_2eSST320.
Not every combination of source and dest is valid.
At least one TOSCA resource must be involved, thus memory to memory is not
supported.
But Tosca resource to Tosca resource is possible, e.g USER to SMEM.
When transferring from VME to VME both must use the same block transfer
mode.
For convenience the two functions toscaDmaWrite() and toscaDmaRead() are provided to transfer from and to memory. These functions call toscaDmaTransfer(). No special DMA enabled memory is required but page aligned memory (e.g. allocated with valloc()) increases efficiency a bit. The Tosca Linux kernel driver takes care of physically fragmented virtual memory and of page locking.
If the swap parameter is 2, 4, or 8, the data is swap byte wise
swapped during transfer, thus allowing to convert between big and little
endian resources.
A positive timeout in milliseconds limits the time the system waits
until a DMA channel becomes available and may result in a timeout error.
Once a DMA transfer has actually started it will not be interrupted by
timeout.
If timeout is negative, the function can block indefinitely if all
DMA channels are reserved and unavailable. On IFC1210, timeouts are not
supported at the moment.
If callback is NULL, the function blocks until the DMA transfer
has completed or times out and returns 0 on success or an error status.
Otherwise the function immediately returns 0 on success or an error code
and (if 0 was returned) starts the DMA in a
worker thread.
It calls the callback function with parameter user and the error
status when the DMA has completed (or timed out) in the context of the
DMA worker thread with the usual multi threading implications:
The callback function is not allowed to block for arbitrary long time.
Keep mutual exclusion in mind becasue the callback function may interrupt
other threads at any time.
Debugging: The global variable toscaDmaDebug can be set to enable
debug output, either to stderr or to toscaDmaDebugFile if that global
FILE* variable is set.
EINVALInvalid combination ofsourceanddestETIMEDOUTTransfer timed outENOMEMOut of memoryEACCESNo permission to use DMA device/dev/dmaproxy*ENOENTDMA device not found
void* toscaDmaLoop();
int toscaDmaLoopsRunning(void);
void toscaDmaLoopsStop();Asynchronous DMA transfers are executed in the context of a DMA worker
thread executing the toscaDmaLoop() function.
This function blocks until a DMA transfers become pending (i.e. the user
called toscaDmaTransfer() with a callback function).
It starts the DMA and waits for completion and then calls the callback
function (which should not block indefinitely).
Then it starts over waiting for the next asynchronous transfer.
The void* result is only provided for compatibility with
pthread_create(). The function only terminates when toscaDmaLoopsStop()
is called and then always returns NULL.
The user is responsible for starting one or more DMA worker threads which execute toscaDmaLoop(). This allows application specific choices for thread parameters like priority and stack size. The stack must be sufficient for any DMA transfer callback function. Starting multiple DMA worker threads may improve throughput because the IFC1210 has and IFC1211 and IFC1410 have four DMA channels which can work in parallel. The EPICS interface starts two four DMA worker threads on an IFC1210 and four otherwise.
The toscaDmaLoopsRunning() function can be used to test how many worker threads are running and toscaDmaLoopsStop() can be used to send the worker threads a signal to terminate. It does not return until all worker threads have stopped.
int toscaIntrConnectHandler(intrmask_t intrmask, void (*function)(), void* parameter);
int toscaIntrDisconnectHandler(intrmask_t intrmask, void (*function)(), void* parameter);
int toscaIntrDisable(intrmask_t intrmask);
int toscaIntrEnable(intrmask_t intrmask);
void toscaInstallSpuriousVMEInterruptHandler(void);These functions allow to connect and disconnect user defined interrupt handler functions to a given USER1, USER2 or VME interrupt and to disable and enable certain interrupt sources. All interrupts start enabled as soon as a handler is connected.
The user function will be called in the context of an
interrupt handler thread with 3 arguments:
The provided parameter, the interrupt number and the interrupt vector
(both int).
The function is not required to use or even accept all three arguments.
The intrmask is a combination of bits that stand for interrupt sources.
The mask structure allows to access multiple interrupt sources at once.
Possible values are bitwise combinations of:
TOSCA_USER1_INTR(n)an interrupt linenin the range from 0 to 15TOSCA_USER1_INTR_MASK(m)a set of interrupts lines according to bits 0 to 15 of mask mTOSCA_USER1_INTR_ANYall 16 USER1 interrupt lines- The same for
USER2 TOSCA_VME_INTR_VEC(n,vec)an interrupt levelnfrom 1 to 7 with vectorvecTOSCA_VME_INTR_MASK_VEC(m,vec)a set of interrupt levels according to bits 0 to 6 of mask m with vectorvecTOSCA_VME_INTR_ANY_VEC(vec)all 7 VME interrupt levels with vectorvecTOSCA_VME_SYSFAILVME system failureTOSCA_VME_ACFAILVME power failureTOSCA_VME_ERRORVME bus errorTOSCA_VME_FAIL(n)withnin the range 0 to 2 meaning one of the above failuresTOSCA_VME_FAIL_ANYall 3 above failuresTOSCA_DEV_*(d,...)same as above with additional device number
It is the number n that will be passed to the interrupt handler
function.
This allows to install the same handler function for multiple
interrupt sources ans still distinguish them.
Using TOSCA_USER1_INTR(n) with n in the range from 16 to 31 is
equivalent to using TOSCA_USER2_INTR(n-16).
Use the VME vector range 1 to 254 only, because vector 255 may be generated in case of errors in the interrupt acknowledge VME bus cycle. Vector 0 means "all vectors" in the functions toscaIntrDisconnectHandler(), toscaIntrDisable() and toscaIntrEnable().
The toscaInstallSpuriousVMEInterruptHandler() function installs a handler
for TOSCA_VME_INTR_ANY_VEC(255) which prints an error message.
The EPICS DevLibVME interface calls this function
automatically the first time VME is used.
Debugging: The global variable toscaIntrDebug can be set to enable
debug output, either to stderr or to toscaIntrDebugFile if that global
FILE* variable is set.
unsigned long long toscaIntrCount();
size_t toscaIntrForEachHandler(size_t (*callback)(const toscaIntrHandlerInfo_t*, void*), void* user);The toscaIntrCount() function returns the total number of interrupts that have been processed since program start.
The toscaIntrForEachHandler() function can be used to iterate over all
installed interrupt handlers.
It calls the callback function for each installed handler until a callback
returns a non zero value.
It returns the result of the last called callback.
It passes two arguments to the callback function:
A pointer to a structure with the fields below (in unspecified order) and
the user argument.
unsigned int device; /* device numner */
intrmask_t intrmaskbit; /* one of the mask bits */
unsigned int index; /* 0...TOSCA_NUM_INTR-1, unique for each intr bit and VME vector */
unsigned int vec; /* 1...255 for VME interrupts, else 0 */
void (*function)(); /* installed handler function */
void *parameter; /* parameter of installed handler function */
unsigned long long count; /* number of times this interrupt has been received */void* toscaIntrLoop();
int toscaIntrLoopIsRunning(void);
void toscaIntrLoopStop();Interrupts are handled in the context of a single thread executing the toscaIntrLoop() function. This function blocks until one of the connected interrupts has been received and then calls the installed interrupt handlers for this interrupt (which should not block indefinitely). Then it starts over waiting for the next interrupt. No two interrupt handler functions will ever execute at the same time.
The user is responsible for starting one interrupt worker thread which executes the toscaIntrLoop() function. This allows application specific choices for thread parameters like priority and stack size. The stack must be sufficient for any installed interrupt handler function. It is not possible to start more than one interrupt handler thread. The EPICS interface starts an interrupt handler thread.
The toscaIntrLoopIsRunning() function returns 1 if the interrupt handler thread is running, else 0. The toscaIntrLoopStop() function sends the interrupt handler thread a signal to terminate. It does not return until the interrupt handler thread has stopped.
int toscaSendVMEIntr(unsigned int level, unsigned int vec);The Tosca VME interface can also generate interrupts on the VME bus to
notify other VME cards.
This is often used together with a VME SLAVE map.
Pass level in the range of 1 to 7 and vec in the range of 0 to 255.
Strictly speaking this is not part of the Tosca API but it is documented here for completeness and helping programmers to convert from pev to Tosca.
int i2cOpen(const char* path, unsigned int address);
int i2cRead(int fd, unsigned int command, unsigned int dlen, void* value);
int i2cWrite(int fd, unsigned int command, unsigned int dlen, int value);The i2cOpen() function takes a path to an I²C bus and the I²C device
address on this bus.
The I²C bus can be given as a device file like "/dev/i2c-2" or simply
as a number "2",
but it is sometimes hard to say which number is assigned by the kernel to
a given I²C bus.
Therefore it is possible to pass a sysfs pattern instead.
The i2cOpen() function uses this glob() compatible pattern to find an
i2c bus. The first match is used and must end in i2c-<number>.
The function returns a file descriptor to be used in i2cRead() and
i2cWrite() functions.
On failure it returns -1 and sets errno.
When done with the device the file descriptor can be closed with close().
Example: The pattern /sys/devices/{,*/}*localbus/*c0.pon-i2c/i2c*
resolves to
/sys/devices/platform/ffe05000.localbus/ffb000c0.pon-i2c/i2c-5 on an
IFC1210 with kernel 4.9 or to
/sys/devices/ffe05000.localbus/ffb000c0.pon-i2c/i2c-5 with older
kernels.
It ends in i2c-5, thus /dev/i2c-5 will be used.
The IFC1210 I²C devices can be found from the Linux shell with:
> ls -d /sys/devices/{,*/}*localbus/*pon-i2c/i2c*
Using a 3.* kernel it returns
/sys/devices/ffe05000.localbus/ffb00080.pon-i2c/i2c-2/
/sys/devices/ffe05000.localbus/ffb000a0.pon-i2c/i2c-3/
/sys/devices/ffe05000.localbus/ffb000b0.pon-i2c/i2c-4/
/sys/devices/ffe05000.localbus/ffb000c0.pon-i2c/i2c-5/
/sys/devices/ffe05000.localbus/ffb000d0.pon-i2c/i2c-6/
/sys/devices/ffe05000.localbus/ffb000e0.pon-i2c/i2c-7/
/sys/devices/ffe05000.localbus/ffb000f0.pon-i2c/i2c-8/
and using a 4.* kernel it returns
/sys/devices/platform/ffe05000.localbus/ffb00080.pon-i2c/i2c-2/
/sys/devices/platform/ffe05000.localbus/ffb000a0.pon-i2c/i2c-3/
/sys/devices/platform/ffe05000.localbus/ffb000b0.pon-i2c/i2c-4/
/sys/devices/platform/ffe05000.localbus/ffb000c0.pon-i2c/i2c-5/
/sys/devices/platform/ffe05000.localbus/ffb000d0.pon-i2c/i2c-6/
/sys/devices/platform/ffe05000.localbus/ffb000e0.pon-i2c/i2c-7/
/sys/devices/platform/ffe05000.localbus/ffb000f0.pon-i2c/i2c-8/
This shows the logical I²C bus numbers (here 2...8) in relation to the hardware address on the processor localbus. To do the opposite, find the hardware addresses of all logical I²C buses, use:
ls -l /sys/bus/i2c/devices/i2c-*
For compatibility with software written for the older pev driver and API, several pev API functions are implemented but actually use the Tosca (or I²C) API.
The global variable pevDebug can be set to enable debug output, either
to stderr or to pevDebugFile if that global FILE* variable is set.
Supported in their pev* and (where existed) in their pevx* form:
- pev(x)_init()
- pev(x)_csr_rd(), pev(x)_csr_wr(), pev(x)_csr_set() (for TCSR and TIO registers)
- pev(x)_elb_rd(), pev(x)_elb_wr() (for PON registers and SRAM)
- pev_smon_rd(), pev_smon_wr() (for Virtex system monitor registers)
- pev_bmr_read(), pev_bmr_write(), pev_bmr_conv_11bit_u(), pev_bmr_conv_11bit_s(), pev_bmr_conv_16bit_u() (for BMR 463 DC/DC regulators, uses I²C API)
- pev(x)_map_alloc(), pev(x)_map_free(), pev(x)_mmap(), pev(x)_munmap(), pev(x)_map_modify() (for memory maps)
- pev(x)_evt_queue_alloc(), pev(x)_evt_queue_free(), pev(x)_evt_register(), pev(x)_evt_read(), pev(x)_evt_queue_enable(), pev(x)_evt_queue_disable(), pev(x)_evt_mask(), pev(x)_evt_unmask() (for interrupts)
- pev(x)_buf_alloc(), pev(x)_buf_free(), pev(x)_dma_move(), pev(x)_dma_status() (for DMA transfers)
Also the wrappers functions we had in the old EPICS pev library are available and use the new Tosca interface:
- pevMap(), pevMapToAddr(), pevMapExt(), pevUnmap()
- pevIntrConnect(), pevIntrDisconnect(), pevIntrEnable(), pevIntrDisable()
- pevDmaAlloc(), pevDmaFree(), pevDmaRealloc()
- pevDmaTransfer(), pevDmaTransferWait(), pevDmaFromBuffer(), pevDmaToBuffer(), pevDmaFromBufferWait(), pevDmaToBufferWait()
Due to fundamental differences between pev and Tosca driver, some functions behave differently.
- pev(x)_map_alloc() cannot be used to map VME SLAVE windows (but pevMap can).
- pev(x)_munmap(), pev(x)_map_free() and pevUnmap() do nothing. The resources are released when the program exits.
- pev(x)_map_modify() always fails.
- pev(x)_evt_register() allocates 256 file descriptors for each VME interrupt level, which may bring the program to the file descriptor limit.
- pev(x)_evt_queue_disable() and pevIntrDisable() simply make the API ignore the interrupts.
- pev(x)_evt_*() and pevIntr*() functions work with Tosca device 0 only.
- pev(x)_buf_alloc() and pevDmaAlloc() simply allocate (page aligned) heap memory, which is sufficient for Tosca DMA.
- pev(x)_dma_move() cannot move from or to PCI other than user space memory.
- pev(x)_dma_status() returns an empty structure because the requested information is not accessible.
The command line utility tosca provides easy access to Tosca
memory maps.
tosca addrspace:offset [wordsize] [bytes]
command | tosca addrspace:offset [wordsize] [bytes]
tosca addrspace:offset [wordsize] [bytes] < file
tosca addrspace:offset [wordsize] [bytes] | command
tosca addrspace:offset [wordsize] [bytes] > file
command $(tosca addrspace:offset [wordsize] [bytes])
command `tosca addrspace:offset [wordsize] [bytes]`
The Tosca resource or address space addrspace is any of TCSR,
TIO, SMEM or SMEM, USER1 or USER, USER2, SRAM,
A16, A24, A32, CRCSR.
The A* address spaces normally access VME in "User Data" mode.
To select "Supervisory" mode add a *, to select "Program" mode add a
# (A16*, A16#, A16*# or A16#*, ...).
The start address offset in bytes as well as the size bytes can be
passed as decimal or hex numbers or in more human readable form with
suffixes k, M, G (for powers of 1024, not case sensitive),
e.g. 1M meaning 0x100000 or 1M3k-80 meaning 0x100bb0.
An offset of :0 may be skipped.
Possible values for wordsize are 1, 2, 4, 8, -2, -4, -8.
Negative wordsize makes the program swap bytes.
In the first form (without redirection of stdin or stdout) tosca
displays the contents of the given address space formatted according to
wordsize. Default wordsize is 2 and default bytes is 0x100.
In the second form (with stdin redirected) tosca copies received data
to the given address space.
If wordsize is negative, the received data is swapped.
If bytes is given and the input size does not match, input is either
truncated or filled with 0 bytes.
In the third form (with stdout redirected) tosca copies data from the
given address space to the output.
If wordsize is negative, the sent data is swapped.
If bytes is given it limits the number of bytes written, otherwise the
output is limited by the address space size or the next MB boundary,
whatever comes first.
Display USER address space with dword swap.
# tosca USER -4 64
0000: 00000003 20160523 5374616e 64617264 .... ..#Standard
0010: 20494f43 20617070 6c696361 74696f6e IOC application
0020: 00000000 00000000 4e6f2046 4d433120 ........No FMC1
0030: 63617264 00000000 4e6f2046 4d433220 card....No FMC2
Read data from USER address space.
# echo $(tosca USER:8 -4 24)
Standard IOC application
Copy range from USER to SMEM offset 1 MB with dword swap.
# tosca USER -4 0x100 | tosca SMEM:1M
Overwrite first 16 bytes of the copy with 0 filled data.
# echo "blabla" | tosca SMEM:1M 1 16
Overwrite more bytes of the copy with data.
# echo "blabla" | tosca SMEM:1M16
Overwrite more bytes of the copy with truncated data.
# echo "blabla" | tosca SMEM:1M32 1 3
Display the copy.
# tosca SMEM:1M 4 64
00100000: 626c6162 6c610a00 00000000 00000000 blabla..........
00100010: 626c6162 6c610a70 6c696361 74696f6e blabla.plication
00100020: 626c6100 00000000 4e6f2046 4d433120 bla.....No FMC1
00100030: 63617264 00000000 4e6f2046 4d433220 card....No FMC2
These functions exist mainly for debug purposes from inside the EPICS IOC shell. The API is usually accessed either by DevLibVME when using EPICS VME device drivers or by the regDev interface. But sometimes it may be helpful to have a more direct access to the API.
See help tosca* on the IOC shell for a list of available functions and
their arguments.
Also many functions print a help text when called without arguments.
To ease access, address space arguments are available by string names.
Possible values are TCSR, TIO, USER1 (or USER), USER2,
SMEM1 (or SMEM), SMEM2, SRAM, A16, A24, A32,
CRCSR.
The A* address spaces normally access VME in "User Data" mode.
To select "Supervisory" mode add a *, to select "Program" mode add a
# (A16*, A16#, A16*# or A16#*, ...).
For VME slave maps use SLAVE.
Any resource address or size can be written in decimal, hexadecimal with
leading 0x or in more human readable form with suffixes k, M, G
(for powers of 1024, not case sensitive).
Also sums and differences are supported.
E.g. 1M meaning 0x100000 or 1M3k-80 meaning 0x100bb0.
Resource addresses are passed as one string in the general format
device:addrspace:address, but 0: for device 0 (the first
and often only Tosca device) can be skipped, as well as :0 to access
address 0 of a given Tosca resource.
Thus 0:USER:0 can be written as USER.
The following examples all use device 0 only and thus skip 0:.
To test mapping of a toscaResource, use:
toscaMap addrspace:address size
It prints the returned pointer (or an error message).
This command can also be used to map a VME SLAVE to
USER, SMEM or program memory.
For automatic swapping add swap.
toscaMap SLAVE:address size USER:address swap
toscaMap SLAVE:address size SMEM:address
toscaMap SLAVE:address size
To display data in memory or on a Tosca resource use the md
(memory display)
function:
md [address](addrspace:) [wordsize] [bytes]
The md command automatically maps Tosca resources with
toscaMap() as necessary.
If addrspace: is not given, the address is in memory, e.g. a
pointer returned by toscaMap.
Default wordsize is 2 with the options 1, 2, 4, 8, -2, -4, -8.
Negative values swap the displayed values.
Default number of bytes is 128.
With no arguments, the function displays the next block of memory using
the previous settings.
All memory maps installed by the running IOC can be listed with
toscaMapShow.
It uses toscaMapForEach() with a function that
prints the map description.
toscaMapShow
addrspace:baseaddr size pointer
TCSR:0x0 0x2000=8K 0xb7b5f000
A32*:0x100000 0x100000=1M 0xb60ae000
SLAVE32:0x100000 0x100000=1M SMEM1:0x0
To test DMA transfers use:
toscaDmaTransfer [addrspace:]sourceaddr [addrspace:]destaddr size [swap]
The possible addrspace values are USER1 (or USER), USER2, SMEM1
(or SMEM), SMEM2 or one of the VME transfer modes A32 (for single
32 bit word transfers), BLT, MBLT, 2eVME, 2eSST160, 2eSST267,
or 2eSST320.
Without addrspace: the address is a memory pointer.
The optional swap parameter is either NS for no swap, WS for
word (2 byte) swap, DS for double word (4 byte) swap, or QS for
quad word (8 byte) swap. The default is no swap.
The function malloc can be used to allocate (page aligned) memory
which can be used here.
It sets the environment variable BUFFER to the start of the allocated
memory, so that commands can use $(BUFFER) to refer to the memory.
malloc 1k
BUFFER = 0xb5fad000
toscaDmaTransfer USER1 $(BUFFER) 1k DS
To get information on interrupt usage, call:
toscaIntrShow [level]
Depending on level, different amount of information is shown.
Level 0, the default, only shows the number of received interrupts since
IOC start and since last toscaIntrShow call for each interrupt source.
Level 1 also lists installed interrupt handler functions and arguments for
each interrupt source.
Level 2 adds the name of the library in which the function was found and
finally level 3 shows the full path of the library.
Negative level numbers have a different meaning. The output is repeated
every -level seconds until a key is pressed.
For example toscaIntrShow -1 repeats every second.
This allows to see interrupt rates.
Only interrupts which have been received since the last output are shown.
The global debug control variables
toscaMapDebug, toscaRegDebug, toscaIntrDebug, and
toscaDmaDebug can be set in the IOC shell with the var command.
Check how many and which Tosca device we have:
> toscaNumDevices
1
> toscaDeviceType 0
1210
> toscaListDevices
0 0000:03:00.0 1210 bridgenum`-1 driververs`0
Map range of 1024 bytes from VME A24 starting at VME address 0x10000 to memory and return pointer:
> toscaMap A24:0x10000 1024
0xb6411000
Display (and automatically map) some VME A16 memory:
> md A16:0x3400
3400: 0000 0000 0000 0000 f800 f800 f800 f800 ................
3410: 0000 0000 0000 0000 0000 0000 0000 0000 ................
3420: 0000 0000 0000 0000 0000 0000 0000 0000 ................
3430: 0000 0000 0000 0000 0000 0000 0000 0000 ................
3440: 0000 0000 0000 0000 0000 0000 0000 0000 ................
3450: 0000 0000 0000 0000 0000 0000 0000 0000 ................
3460: 0000 0000 0000 0000 0000 0000 0000 0000 ................
3470: 0000 0000 0000 0000 0000 0000 0000 0000 ................
Display (and map) USER, 4 byte wise swapped as needed by the application in USER:
> md USER -4
0000: 00000003 20160523 5374616e 64617264 .... ..#Standard
0010: 20494f43 20617070 6c696361 74696f6e IOC application
0020: 00000000 00000000 4e6f2046 4d433120 ........No FMC1
0030: 63617264 00000000 4e6f2046 4d433220 card....No FMC2
0040: 63617264 00000000 00000000 00000000 card............
0050: 00000000 00000000 00000000 00000000 ................
0060: 00000000 00000000 00000000 00000000 ................
0070: 00000000 00000000 00000000 00000000 ................
Map VME A32 slave window at 3MB to first 256kB USER:
> toscaMap SLAVE:3M 256k USER swap
Success
Display above VME slave window to USER with wrong 2 byte access:
> md A32:3M
00300000: 0003 0000 0523 2016 616e 5374 7264 6461 .....# .anStrdda
00300010: 4f43 2049 7070 2061 6361 6c69 6f6e 7469 OC Ipp acalionti
00300020: 0000 0000 0000 0000 2046 4e6f 3120 4d43 ........ FNo1 MC
00300030: 7264 6361 0000 0000 2046 4e6f 3220 4d43 rdca.... FNo2 MC
00300040: 7264 6361 0000 0000 0000 0000 0000 0000 rdca............
00300050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00300060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00300070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
Display above VME slave window to USER with correct 4 byte access: (Note that swapping is handled by the SLAVE window.)
> md A32:3M 4
00300000: 00000003 20160523 5374616e 64617264 .... ..#Standard
00300010: 20494f43 20617070 6c696361 74696f6e IOC application
00300020: 00000000 00000000 4e6f2046 4d433120 ........No FMC1
00300030: 63617264 00000000 4e6f2046 4d433220 card....No FMC2
00300040: 63617264 00000000 00000000 00000000 card............
00300050: 00000000 00000000 00000000 00000000 ................
00300060: 00000000 00000000 00000000 00000000 ................
00300070: 00000000 00000000 00000000 00000000 ................
Lookup existing maps (of this IOC):
> toscaMapShow
addrspace:baseaddr size pointer
TCSR:0x0 0x2000=8K 0xb7eb2000
A24:0x0 0x100000=1M 0xb6401000
A16:0x0 0x10000=64K 0xb63f1000
USER1:0x0 0x100000=1M 0xb62f1000
SLAVE32:0x300000 0x100000=1M USER1:0x0 SWAP
A32:0x300000 0x100000=1M 0xb61f1000
Do 128 kB DMA transfer from USER to memory with double word swap and display the transferred data:
> malloc 128k
BUFFER = 0xb5eea000
> toscaDmaTransfer USER:0x100 $(BUFFER) 128k DS
Success
> md $(BUFFER) 4
b5eea000: 00000003 20160523 5374616e 64617264 .... ..#Standard
b5eea010: 20494f43 20617070 6c696361 74696f6e IOC application
b5eea020: 00000000 00000000 4e6f2046 4d433120 ........No FMC1
b5eea030: 63617264 00000000 4e6f2046 4d433220 card....No FMC2
b5eea040: 63617264 00000000 00000000 00000000 card............
b5eea050: 00000000 00000000 00000000 00000000 ................
b5eea060: 00000000 00000000 00000000 00000000 ................
b5eea070: 00000000 00000000 00000000 00000000 ................
> md
b5eea080: 00000000 00000000 00000000 00000000 ................
b5eea090: 00000000 00000000 00000000 00080000 ................
b5eea0a0: 00080000 00080000 00080000 00080000 ................
b5eea0b0: 00080000 00080000 00080000 00080000 ................
b5eea0c0: 00080000 00080000 00080000 00080000 ................
b5eea0d0: 00080000 00080000 00080000 00000000 ................
b5eea0e0: 00000000 00000000 00000000 00000000 ................
b5eea0f0: 00000000 00000000 00000000 00000000 ................
Try 1 MB DMA transfer from A32 to SMEM using MBLT mode from an address where no MBLT capable VME card is available:
> toscaDmaTransfer MBLT:2M SMEM:0 1M
Input/output error
Check for VME errors and re-arm error catching:
> toscaGetVmeErr
0x00200000,0x98000805 (ERR R TOUT DMA id=4 len=0 MBLT:0x200000)
Connect some interrupts:
> toscaIntrConnectHandler USER1-1-4;8 toscaDebugIntrHandler
> toscaIntrConnectHandler VME-4.18 toscaDebugIntrHandler
> toscaIntrConnectHandler VME.20 toscaDebugIntrHandler
Check interrupts:
> toscaIntrShow 1
total number of interrupts: 0 (+0)
VME-4.18 count=0 (+0) toscaDebugIntrHandler(0x100d9e50)
VME-1.20 count=0 (+0) toscaDebugIntrHandler(0x100d9ab0)
VME-2.20 count=0 (+0) toscaDebugIntrHandler(0x100d9ab0)
VME-3.20 count=0 (+0) toscaDebugIntrHandler(0x100d9ab0)
VME-4.20 count=0 (+0) toscaDebugIntrHandler(0x100d9ab0)
VME-5.20 count=0 (+0) toscaDebugIntrHandler(0x100d9ab0)
VME-6.20 count=0 (+0) toscaDebugIntrHandler(0x100d9ab0)
VME-7.20 count=0 (+0) toscaDebugIntrHandler(0x100d9ab0)
USER1-1 count=0 (+0) toscaDebugIntrHandler(0x100d9c10)
USER1-2 count=0 (+0) toscaDebugIntrHandler(0x100d9c10)
USER1-3 count=0 (+0) toscaDebugIntrHandler(0x100d9c10)
USER1-4 count=0 (+0) toscaDebugIntrHandler(0x100d9c10)
USER1-8 count=0 (+0) toscaDebugIntrHandler(0x100d9c10)
> toscaIoRead 0
0x805d0910
> toscaCsrRead 0
0x805d0910
> toscaRead CSR:0
0x805d0910
> md TCSR -4 16
0000: 805d0910 000000ba 00000000 00030000 .]..............
> toscaRead USER:4
0x20160523
> toscaRead SHM:1M
0xdd580115
> toscaWrite SHM:1M 0xdeadbeef
0xdeadbeef
> toscaClear SHM:1M 0xff0000ff
0x00adbe00
> toscaSet SHM:1M 0x5555aaaa
0x55fdbeaa
> toscaPonRead 0x1c
signature 0x04042016
> toscaPonRead
0x00 vendor 0x73571210
0x04 static_options 0x00000910
0x08 vmectl 0x3000ff7e
0x0c mezzanine 0xc077f703
0x10 general 0xffffff98
0x14 pciectl 0x00010201
0x18 user 0x00000000
0x1c signature 0x04042016
0x20 cfgctl 0x80010707
0x24 cfgdata 0x00000000
0x40 bmrctl 0x00000000
Read Virtex system monitor registers:
>toscaSmonRead 0x08
Supply offs 0xfe67 = 2.979 V
>toscaSmonRead
0x00 Temp 0xb341 = 79.73 C
0x01 Vccint 0x55b0 = 1.002 V
0x02 Vccaux 0xddc3 = 2.599 V
0x03 Vadj 0x0000 = 0.000 V
0x04 VrefP 0x0000 = 0.000 V
0x05 VrefN 0x0000 = 0.000 V
0x08 Supply offs 0xfe64 = 2.979 V
0x09 ADC offs 0xfeae = 2.982 V
0x10 Vaux[0] 0x0000 = 0.000 V
0x11 Vaux[1] 0x0000 = 0.000 V
0x12 Vaux[2] 0x0000 = 0.000 V
0x13 Vaux[3] 0x0000 = 0.000 V
0x14 Vaux[4] 0x0000 = 0.000 V
0x15 Vaux[5] 0x0000 = 0.000 V
0x16 Vaux[6] 0x0000 = 0.000 V
0x17 Vaux[7] 0x0000 = 0.000 V
0x18 Vaux[8] 0x0000 = 0.000 V
0x19 Vaux[9] 0x0000 = 0.000 V
0x1a Vaux[A] 0x0000 = 0.000 V
0x1b Vaux[B] 0x0000 = 0.000 V
0x1c Vaux[C] 0x0000 = 0.000 V
0x1d Vaux[D] 0x0000 = 0.000 V
0x1e Vaux[E] 0x0000 = 0.000 V
0x1f Vaux[F] 0x0000 = 0.000 V
0x20 Temp Max 0xb395 = 80.22 C
0x21 Vccint Max 0x56fc = 1.017 V
0x22 Vccaux Max 0xde22 = 2.602 V
0x24 Temp Min 0xaf69 = 71.86 C
0x25 Vccint Min 0x5544 = 0.999 V
0x26 Vccaux Min 0xdd57 = 2.593 V
0x3f Flag 0x0000 = 0000:0000:0000:0000
0x40 Config #0 0x0000 = 0000:0000:0000:0000
0x41 Config #1 0x0000 = 0000:0000:0000:0000
0x42 Config #2 0x0800 = 0000:1000:0000:0000
Read FMC registers over serial bus:
> toscaSbcRead 1 0x1000000
0x44469758
The devLibVME interface is automatically initialized when this EPICS driver is loaded. It implements the standard EPICS VME bus access functions for Tosca device 0 (the first and often only Tosca device). Thus any existing EPICS driver using functions from devLibVME.h should be able to access the VME bus using the Tosca VME interface. (However older drivers may need a conversion from VxWorks to the EPICS osi (operating system independent) support.)
The EPICS devLibVME interface has no means to select "User" or "Supervisory" mode, nor "Data" or "Program" mode for VME memory maps. To be compatible with what we had in vxWorks, the used mode is fixed to "Supervisory Data" on A16, A24 and A32. This was done because some VME cards allow access only in "Supervisory" mode and most others ignore this flag.
Interrupt handlers can be installed for any VME interrupt vector 1 to 255. However 255 may be the result of a problem and should not be used in applications. This driver automatically installs handlers for spurious interrupt detection for interrupt vectors 255.
Tosca handles VME interrupts per combination of interrupt level (1-7) and
vector.
But the EPICS devLibVME interface allows to install handlers only per
vector.
Thus this driver always installs a handler for a given interrupt vector
for all 7 interrupt levels.
The interrupt handlers are called in the context of the
thread "irq-TOSCA".
The EPICS osi priority of this thread is 80 by default but can be set with
the IOC shell variable toscaIntrPrio (before iocInit).
The devLibVME functions devLibA24Malloc() and devLibA24Free() are unsupported (devLibA24Malloc() always returns NULL) because Tosca does not support VME A24 slave windows.
The function devInterruptInUseVME() always returns FALSE, because the driver can handle a list of interrupt handlers for each interrupt vector. The function is supposed to return TRUE if the vector cannot be used any more (I think).
Two or four (depending on the Tosca device type)
DMA worker threads "dma*-TOSCA" are
started automatically (even though devLibVME has no DMA support).
The EPICS osi priority of these threads is 80 by default but can be set
with the IOC shell variable toscaDmaPrio (before iocInit).
Debugging: Debug messages can be enabled by setting the IOC shell
variable toscaDevLibDebug to 1.
Also all other tosca debug variables like toscaMapDebug are available
from the IOC shell.
The generic regDev device support is meant to allow EPICS records access to "registers" on almost any type of hardware. It has support for all hardware accessing record types from EPICS base, including the array records aai and aao. For details see the regDev manual.
This driver makes all the mappable Tosca resources (USER1, USER2, SMEM1, SMEM2, TCSR, TIO, SRAM, VME A16, A24, A32, CRCSR and VME SLAVE) as well as the PON and SMON registers available to regDev.
The I²C devices on the IFC1210 are available through regDev as well. However the I²C devices are not implemented by the Tosca driver but by the i2cDev driver using the API described above. This allows to access all resources on IFC boards in a unified way.
In addition, DMA is supported for transfers from and to USER1, USER2, SMEM1, SMEM2 and VME A32, which will be used for large arrays or in regDev's block mode.
DMA transfers from and to VME can be implemented with different VME transfer modes, but the availability depends on the accessed VME card. Not all cards support all modes. The default DMA mode is SCT (single cycle transfer). It is most portable because it does not use any VME block transfer mode but only 32 bit accesses to the A32 address space. It is also the slowest mode.
Measured VME transfer speeds in MB/sec using VME SLAVE map on the same IFC1210.
| VME DMA mode | read | write |
|---|---|---|
| memory map 32 bit | 2 | 14 |
| SCT | 7 | 19 |
| BLT | 25 | 36 |
| MBLT | 48 | 61 |
| 2eVME | 100 | 93 |
| 2eSST160 | 110 | 95 |
| 2eSST267 | 149 | 119 |
| 2eSST320 | 169 | 137 |
require "tosca"
toscaRegDevConfigure name addrspace:address size flags
toscaSbcDevConfigure name fmc_slot address size
toscaSmonDevConfigure name
toscaPonDevConfigure name
The toscaRegDevConfigure function creates a new logical regDev device
with the given name which maps to an address range
starting at addrspace:address with the given size in bytes.
It is possible to create multiple regDev devices on the same Tosca
resource, even with overlapping ranges, which can make sense if the
flags differ.
The offsets used in the record links are bytes relative to the start
address of the address range.
For addrspace use one of USER1 (or USER), USER2, SMEM1 (or
SMEM), SMEM2, TCSR, TIO, SRAM, A16, A24, A32, CRCSR.
Add * for "Supervisory" and # for "Program" access to the A
modes.
Possible flags are:
- swapping
NSno swapWS,WL,WBword (2 byte) swap, little endian, big endianDS,DL,DBdouble word (4 byte) swap, little endian, big endianQS,QL,QBquad word (8 byte) swap, little endian, big endian
- VME DMA modes (only for DMA on
A32address space)SCT"single cycle transfer", the defaultBLTVME 32 bit block transferMBLTVME 64 bit block transfer2eVMEdouble edge VME 64 bit block transfer2eSST160double edge source synchronous transfer max 160 MB/sec2eSST267double edge source synchronous transfer max 267 MB/sec2eSST320(or short2eSST) double edge source synchronous transfer max 320 MB/sec
- block mode (transfer full address range with DMA)
blockuse block mode for reading and writingblockreaduse block mode for readingblockwriteuse block mode for writing
- DMA limits for arrays (minimum number of elements)
dmaReadLimit= default 100dmaWriteLimit= default 2kdmaonlysets both limits to 1nodmasets both limits to 0
- default interrupt vector (if not set in the record)
intr= 1...254 for VME, 0-15 for USER1, USER2
The *L and *B swap modes define the endianess of the accessed
resource independently of the endianess of the CPU. Thus *L modes
(for little endian resources) only swap on big endian CPUs and likewise
the *B modes only on little endian CPUs.
For USER1, USER2, TSCR and TIO address space the default is DL,
for all others (including SMEM1 and SMEM2) it is NS.
The driver transfers data either by reading and writing through a
memory map or using DMA tranfers.
DMA is used for reading or writing arrays if the number of elements is at
least dmaReadLimit or dmaWriteLimit respectively.
The default values are chosen at the break even point, where DMA becomes
faster than memory mapped access.
This point depends on the time needed to set up a DMA transfer compared to
the memory mapped read and write speed.
Setting the limit to 1 uses DMA for any transfer.
If both limits are 1 (e.g. using dmaonly) no memory map is created.
If both limits are 0 (e.g. using nodma) DMA is never used.
To access to FMC registers over the serial bus interface use toscaSbcDevConfigure() with the FMC number (1 or 2) and the base address of the FMC component. For the ADC/DAC 311x family base addresses are listed in the FMC device registers chapter. For details refer to the documentation of the FMC module in use.
SMON and PON register access need no
configuration parameters and have fixed size,
thus the only parameter to pass to toscaSmonDevConfigure and
toscaPonDevConfigure is a name to be used in the record links.
The block mode treats the whole device as one large array which is transferred to or from memory when a record with PRIO=HIGH reads from or writes to the device. This allows all records, not only arrays, to benefit from the speed of a single DMA transfer which gives a performance boost when reading many values at the same time. Any other records accessing the same device which do not have PRIO=HIGH only read from or write to the buffer. Make sure that the record which triggers the DMA read is processed first and the record which triggers the DMA write is processed last.
DMA block read may be combined with interrupt triggered processing. The device (FPGA USER or a VME device) may generate an interrupt when new data is available. The record triggering the DMA read can use the "V=vector" flag in its INP link and SCAN="I/O Intr" to enable interrupt processing. For VME vector is 1-254, for USER or SMEM vector is 0-15. (SMEM devices use USER interrupts).
Block mode can be enabled separately for reading and writing with
blockread and blockwrite, block is just a shortcut for both.
See also the regDev documentation.
field (DTYP, "regDev")
field (INP, "@name:offset T=datatype [flags]")
field (OUT, "@name:offset[:initoffset] T=datatype [flags]")
Most Tosca resouces assume a 32 bit datatype like int32 or uint32.
The Virtex Smon registers use uint16.
For other choices and flags see the regDev documentation.
For calculating human readable values from register readings (in particular scaling from integer to floating point) refer to the respective hardware documentation.
field (SCAN, "I/O Intr")
field (INP, "@name:offset T=datatype V=[vector] [flags]")
Here vector is the VME interrupt vector (1-254) for VME devices or
the USER1 or USER2 interrupt line (0-15) for FPGA devices.
If the device is a SMEM1 or SMEM2 map, the interrupt lines from
USER1 or USER2 respectively are used.
Output records can be triggered by interrupts as well but this is quite uncommon.
In the startup script define some devices:
toscaRegDevConfigure user USER1:0
toscaSmonDevConfigure virtex
toscaRegDevConfigure sharedmem SHM:1M 64k DL
toscaSbcDevConfigure fmc1-lmk 1 0x02000000 256
Create some records
record (longin, "$(PREFIX)USER-BUILDDATE)
{
field(DTYP, "regDev")
field(INP, "@user:4 T=uint32")
field(PINI, "YES")
}
record (stringin, "$(PREFIX)USER-LABEL")
{
field(DTYP, "regDev")
field(INP, "@user:0x08 L=32")
field(PINI, "YES")
}
record (ai, "$(PREFIX)VIRTEX-Temp")
{
field(DTYP, "regDev")
field(INP, "@virtex:0 T=uint16")
field(LINR, "LINEAR")
field(EGUF, "230.825")
field(EGUL, "-273.15")
field(PREC, "2")
field(EGU, "C")
field(SCAN, "1 second")
}
record (ai, "$(PREFIX)VIRTEX-VccInt")
{
field(DTYP, "regDev")
field(INP, "@virtex:1 T=uint16")
field(LINR, "LINEAR")
field(EGUF, "3")
field(PREC, "3")
field(EGU, "V")
field(SCAN, "1 second")
}
record (aai, "$(PREFIX)DATA")
{
field(DTYP, "regDev")
field(INP, "@smem:0 T=uint32 V=5")
field(SCAN, "I/O Intr") # trigger with intr USER1-5
field(NELM, "4096")
field(LOPR, "-10")
field(HOPR, "10")
field(PREC, "6")
}
record (longout, "$(PREFIX)FMC1-ADC3110-ClkOut_3")
{
field(DTYP, "regDev")
field(OUT, "fmc1-lmk:0x03 T=uint32")
}
For backward compatibility with the older pev driver, the old startup script functions pev(Asyn)Configure and pev(Asyn)I2cConfigure are still available but call Tosca or i2cDev configuration functions (and print to the shell how they can be replaced). Also the resource names used by pev (e.g. "SH_MEM") are still understood by Tosca. No change is needed in the record configuration because both use regDev.
The transition from pev(Asyn)Configure to toscaRegDevConfigure is quite straight forward:
# pev(Asyn)Configure card name addrspace address DMA_mode vec size block swap vmePktSize
toscaRegDevConfigure name card:addrspace:address size flags
The name is now the first argument but the meaning stays the same.
Second argument is now card:addrspace:address as one string.
Card 0: and address :0 can be skipped.
Third argument is now size.
All further arguments are flags in arbitrary order.
The existing DMA_modes and swap choices are understood.
The block argument is either the string block or nothing.
The interrupt vector vec can be passed as intr=vec, but now it is
possible that different records use different interrupts vectors with the
option V= in the link.
Using this method, interrupt triggered records can now also be used on
block devices.
The vmePktSize option is not supported any more.
The pev driver automatically created a regDev device for a TSCR memory map with the name "pev_csr". The Tosca driver does the same to stay compatible.
The transition from pev(Asyn)I2cConfigure to i2cDev needs a bit more work but also gives some benefit.
# pev(AsynI)2cConfigure card name i2cControlWord command
i2cDevConfigure name sysfspattern device muxdev=val
Again, name has the same meaning as before.
The PON FPGA to which the I²C buses are connected is connected to the
localbus of the processor.
Hence only I²C buses on card 0 are accessible and there is no card
argument any more.
The bus and device address of the I²C device used to be coded in the
i2cControlWord.
The highest 3 bits describe the bus number and the lowest 7 bits the
device address.
Now, an I²C bus can be identified using either a /dev/i2c* device file
or simply a number or, because it is hard to tell in advance which hardware
bus has which number, a sysfs pattern.
See the I²C API for possible sysfs patterns.
The bus sysfs pattern can be derived from the highest hex digit of the control word:
| i2cControlWord | sysfspattern | connected hardware |
|---|---|---|
| 0x00000000 - 0x1FFFFFFF | /sys/devices/{,*/}*localbus/*80.pon-i2c/i2c* |
temperature sensors |
| 0x40000000 - 0x5FFFFFFF | /sys/devices/{,*/}*localbus/*a0.pon-i2c/i2c* |
power monitoring |
| 0x60000000 - 0x7FFFFFFF | /sys/devices/{,*/}*localbus/*b0.pon-i2c/i2c* |
transition card over P0 |
| 0x80000000 - 0x9FFFFFFF | /sys/devices/{,*/}*localbus/*c0.pon-i2c/i2c* |
XMC1/FMC1 slot |
| 0xA0000000 - 0xBFFFFFFF | /sys/devices/{,*/}*localbus/*d0.pon-i2c/i2c* |
XMC2/FMC2 slot |
| 0xC0000000 - 0xDFFFFFFF | /sys/devices/{,*/}*localbus/*e0.pon-i2c/i2c* |
PCIe switch |
| 0xE0000000 - 0xFFFFFFFF | /sys/devices/{,*/}*localbus/*f0.pon-i2c/i2c* |
programmable oscillator |
The i2cDevConfigure function allows definition of multiplexer (mux)
settings.
This ensures that the mux programming is integrated in the device access in
a thread safe way and cannot be changed accidentally by a different thread
trying to access another device with different mux settings on the same I²C
bus at the same time.
The mux devices in use are programmed with a single byte command, so that
they can be defined as muxdevval= with muxdev being the device
address of the mux device and val the value sent to the mux device.
Of course this change requires to modify the templates and to remove the
mux setting records.
The old method using separate records to program the mux devices still
works but is not thread save and thus not recommended.
However, be aware that this mechanism is not process save, only thread
save.
Other processes may re-program the mux while this driver uses it.
To get a process save way to program mux devices, they must be handled by
the Linux kernel and therefore need to be defined in the system device
tree.
This may be difficult for mux devices on pluggable hardware like
transition cards.
Example (a device that requires mux 0x70 to set be to 0 and mux 0x71 to 8):
# pevAsynI2cConfigure 0 gtx1143_A0 0x60000050
i2cDevConfigure gtx1143_A0 /sys/devices/{,*/}*localbus/*b0.pon-i2c/i2c* 0x50 0x70`0 0x71`0x8
The ifc1210 device support had covered access to PON and SRAM (under
the common name of ELB), to SMON, and the BMR 463 DC/DC regulators.
Now all are accessible through regDev, using toscaPonDevConfigure,
toscaSmonDevConfigure, and toscaRegDevConfigure with SRAM address
space.
While SRAM access through ELB needed an address offset of 0xe000, the
access through toscaRegDevConfigure does not use such an offset.
The BMR 463 DC/DC regulators are actually I²C devices and as such
accessible with i2cDevConfigure:
toscaPonDevConfigure PON
toscaSmonDevConfigure SMON
toscaRegDevConfigure SRAM SRAM 8k
i2cDevConfigure BMR0 "/sys/devices/{,*/}*localbus/*a0.pon-i2c/i2c*" 0x53
i2cDevConfigure BMR1 "/sys/devices/{,*/}*localbus/*a0.pon-i2c/i2c*" 0x5b
i2cDevConfigure BMR2 "/sys/devices/{,*/}*localbus/*a0.pon-i2c/i2c*" 0x63
i2cDevConfigure BMR3 "/sys/devices/{,*/}*localbus/*a0.pon-i2c/i2c*" 0x24
The split between "Main" and "Target" configuration has been removed. VME SLAVE "Target" maps can be defined with 1MB granularity anywhere in the first 256 MB of the A32 address space. There are two methods to define SLAVE maps to USER or SMEM. Either create a SLAVE map from the IOC shell that will be deleted when the IOC terminates. Or a create a persistent map from the Linux command line that cannot be deleted.
In the IOC shell (e.g. IOC startup script) use the toscaMap function:
# pevVmeSlaveMainConfig AM32 mainbase mainsize
# pevVmeSlaveTargetConfig AM32 targetbase targetsize "" resource resource_address [AUTO]
toscaMap SLAVE:mainbase+targetbase targetsize resource:resource_address [SWAP]
Simply add the targetbase to the mainbase to the VME address.
The resource names can be copied using the pev style names or replaced
with the Tosca style names, the driver understands both.
Example
# pevVmeSlaveMainConfig("AM32", 0x01000000, 0x01000000)
# pevVmeSlaveTargetConfig("AM32", 0x000000, 0x100000, "", "SH_MEM", 0x000000, "AUTO")
# pevVmeSlaveTargetConfig("AM32", 0x100000, 0x100000, "", "USR1" , 0x000000, "AUTO")
toscaMap SLAVE:0x01000000 1M SMEM SWAP
toscaMap SLAVE:0x01100000 1M USER SWAP
On the Linux command line use the sys file system:
echo vme_address:size:resource_address:resource:swap > '/sys/class/vme_user/bus!vme!s0/add_slave_window'
Pass vme_address, size and resource_address in hex.
The resource name can be either pci, vme, usr or shm.
The swap argument can be either yes or no.
(Actually anything starting with y or Y means yes, anything
else means no.)
Example
echo 0x1000000:0x100000:0:shm:yes > '/sys/class/vme_user/bus!vme!s0/add_slave_window'
echo 0x1100000:0x100000:0:usr:yes > '/sys/class/vme_user/bus!vme!s0/add_slave_window'