BatchMarc we process marc files on go + js
All beaver!
Today is Friday, and that means time to sum up!
Continuing the themes of our library club, I hasten to please everyone a little bit: I did it!
After a sea of spent nerve cells, after hell of documentation on marc formats, and a bunch of crutches, I still collected a relatively sane (but so far with terrible code :)) converter (+ handler + everything that is enough imagination) for marc formats.
')
https://github.com/HerzenLibRu/BatchMarc (handler, rules are written to js)
https://github.com/t0pep0/marc21 (Go library for working with marc formats)
You can run like this:
go run main.go inputFile.ldb outputFile.ldb rules.js
The rules apply to each marc entry in the file.
Documentation has not yet been written, but is everything ahead of us? =)
Some library nuances (and the converter, respectively):
Features of writing rules:
When your rule starts to run, there are already two objects: orig and res - orig - the original entry, res - what will eventually be written to the output file, and when you start your rule, res is an empty object just initialized, while orig filled with data
When your rules work, Go will take the data from res, and write it to a file.
Attention! The code is not optimized, consider that this is the minimum viable product.
An example of the rules can be found in the BatchMarc repository.
PullRequest, IssueRequest - accepted
UPD: js for writing the rules is chosen as the most widely distributed (in the library environment, including) language
Today is Friday, and that means time to sum up!
Continuing the themes of our library club, I hasten to please everyone a little bit: I did it!
After a sea of spent nerve cells, after hell of documentation on marc formats, and a bunch of crutches, I still collected a relatively sane (but so far with terrible code :)) converter (+ handler + everything that is enough imagination) for marc formats.
')
https://github.com/HerzenLibRu/BatchMarc (handler, rules are written to js)
https://github.com/t0pep0/marc21 (Go library for working with marc formats)
You can run like this:
go run main.go inputFile.ldb outputFile.ldb rules.js
The rules apply to each marc entry in the file.
Documentation has not yet been written, but is everything ahead of us? =)
Some library nuances (and the converter, respectively):
- subfields are stored in a linked list with sorting when added (specific requests from colleagues did not allow using hashmap)
- Leader flag naming may not please you
Features of writing rules:
When your rule starts to run, there are already two objects: orig and res - orig - the original entry, res - what will eventually be written to the output file, and when you start your rule, res is an empty object just initialized, while orig filled with data
function VariableSubField(name, data){ this.Name = name; this.Data = data; } function VariableField(tag, indicatorOne, indicatorTwo) { this.Tag = tag; this.IndicatorOne = indicatorOne; this.IndicatorTwo = indicatorTwo; this.RawData = []; this.SubField = []; } function Leader() { this.Status = ""; this.Type = ""; this.BibLevel = ""; this.ControlType = ""; this.CharacterEncoding = ""; this.IndicatorCount = ""; this.SubfieldCodeCount = ""; this.EncodingLevel = ""; this.CatalogingForm = ""; this.MultipartLevel = ""; this.LengthOFFieldPort = ""; this.StartCharPos = ""; this.LengthImplemenDefine = ""; this.Undefine = ""; } function MarcRecord(){ this.Leader = new Leader() this.VariableField = [] }; orig = new MarcRecord(); res = new MarcRecord(); When your rules work, Go will take the data from res, and write it to a file.
Attention! The code is not optimized, consider that this is the minimum viable product.
An example of the rules can be found in the BatchMarc repository.
PullRequest, IssueRequest - accepted
UPD: js for writing the rules is chosen as the most widely distributed (in the library environment, including) language
Source: https://habr.com/ru/post/395377/
All Articles