File system access in MarkLogic Server

MarkLogic is an application server and any program written in XQuery for it can get access not only to objects stored in the database itself, but also to files located directly on the file system.
The API that provides access to the file system in MarkLogic Server is not so rich, but the available tools are enough to read the data from the file system directly from the XQuery code and save the files to it.

As part of the MarkLogic API, an external file or object is a file or object stored on the file system. Accordingly, an internal object is an object that is stored in the database itself.

Let's consider in more detail the existing API. The file system access functions are located in the “xdmp” namespace.
And the first function allows you to access the binary () object lying on the file system

xdmp:external-binary( $path as xs:string, [$starting-location as xs:double], [$length as xs:double] ) as binary() 

$ path - the path to the file
$ starting-location - The index of the first byte in the file is 1. Default = 1
$ length - The number of bytes read.
In this case, the binary () object is associated with the file in FS and you can always determine if the binary () object is internal or external.
You must have privileges to access this feature.
marklogic.com/xdmp/privileges/xdmp-external-binary
If the file on the file system does not exist, then the XDMP-MISSINGFILE exception is thrown.
')
The following function takes a binary () file object and returns the path to the file associated with it on the file system.

 xdmp:external-binary-path( $source as binary() ) as xs:string? 

The XDMP-ARG exception is thrown if the object passed to the binary () function is not associated with a file on the file system.

The xdmp function : binary-is-external - checks if a binary () object is an external object (file)

 xdmp:binary-is-external( $source as binary() ) as xs:boolean 

Using the following function, you can get a listing of the directory on the file system.

 xdmp:filesystem-directory( $pathname as xs:string ) as element(dir:directory) 

$ pathname - directory of interest
To perform the function, you must have rights
marklogic.com/xdmp/privileges/xdmp-filesystem-directory
Also, the system user must have read access to the specified directory.
Exceptions SVC-DIROPEN or SVC-FILOPEN are thrown out if the user performing the function does not have sufficient rights to do this.
The result of the function is a list of objects located on the FS. Example

 xdmp:filesystem-directory( "./" ) <dir:directory xsi:schemaLocation="http://marklogic.com/xdmp/directory directory.xsd" xmlns:dir="http://marklogic.com/xdmp/directory" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <dir:entry> <dir:filename>Admin</dir:filename> <dir:pathname>./Admin</dir:pathname> <dir:type>directory</dir:type> <dir:content-length>0</dir:content-length> <dir:last-modified>2013-05-02T13:09:53+04:00</dir:last-modified> </dir:entry> <dir:entry> <dir:filename>LEGALNOTICES.txt</dir:filename> <dir:pathname>./LEGALNOTICES.txt</dir:pathname> <dir:type>file</dir:type> <dir:content-length>28343</dir:content-length> <dir:last-modified>2013-04-19T23:06:32+04:00</dir:last-modified> </dir:entry> ... </dir:directory> 

To read text data from the file system, you can use this function.

 xdmp:filesystem-file( $pathname as xs:string ) as xs:string 

Where $ pathname is the path to the file to be read in XQuery
In order for the read operation to end successfully, it is necessary that the data in the file be in UTF-8 encoding. Because of this requirement, when reading binary data, this function will throw an exception and MarkLogic will vouch for the data encoding. To read binary data, use the xdmp function : external-binary .
Privileges are required to perform the function.
marklogic.com/xdmp/privileges/xdmp-filesystem-file

You can check whether the file exists on the file system

 xdmp:filesystem-file-exists( $pathname as xs:string ) as xs:boolean 

Where $ pathname is the path to check
Privileges are required to perform the function.
marklogic.com/xdmp/privileges/xdmp-filesystem-file-exists

In order to find out the file size, there is an xdmp function : filesystem-file-length

 xdmp:filesystem-file-length( $pathname as xs:string ) as xs:unsignedLong? 

The function returns an empty sequence if the file does not exist.
Privileges required to perform the function.
marklogic.com/xdmp/privileges/xdmp-filesystem-file-length

Next, consider the function of saving

 xdmp:save( $path as xs:string, $node as node(), [$options as node()?] ) as empty-sequence() 

This function serializes any (xml, text, bunary) object and saves it to the file system under the specified name in the specified directory.
This function has quite a few options, for example, output-encoding, which specifies the document encoding. But the existing options of this function are quite specific and in order to simply save the document to the file system, it suffices to execute the following code:

 let $text := text { "hello world" } return xdmp:save("greeting.txt", $text) 

or for example, you can save a file from DB to the file system

 let $pdf := doc("/mydocs/stuff.pdf") return xdmp:save("mystuff.pdf", $pdf) 

Privileges required to perform the function.
marklogic.com/xdmp/privileges/xdmp-save

MarkLogic Server is primarily storing documents in a database and processing these documents with XQuery queries. But sometimes you need to read or write the file to the file system, and for this MarkLogic has everything you need. Of course, these tools are not suitable for complex processing of binary files, image conversions, but they do an excellent job with the tasks set before them.