Ejabberd 14.x on FreeBSD 10: Installation and Basic Setup
Good day!
A small preface: the 13th saw the light not so long ago, and the 14th version of ejabberd followed this year. Process One decided to change the versioning scheme for a number of reasons, and version 2.x is now considered obsolete. The new ejabberd shared on two branches - ejabberd Community Server and ejabberd Business Edition and began to develop quite actively.
In our case, FreeBSD was chosen as the system, since it is necessary to maintain a fairly large number of connections per node (~ 100k).
')
I don’t see any reason to stop installing the system, so under the cut we will immediately proceed to the setting.
Some ejabberd data will be stored in mysql, so odbc libraries must be installed on the system. We also use the external authorization script in php, so php is also needed.
Stage 1:
Put the prerequisites (git, fop, libyaml, wget, unixODBC, expat, wx30-gtk2, php5, etc):
Downloading the latest version of Erlang (at the time of writing of article 17.2.2):
We collect, we put it:
Next, download ejabberd and ejabberd-contrib:
Create user ejabberd, specify / var / lib / ejabberd as the home directory.
We compile and install ejabberd:
When building erlang and ejabberd, various nuances may arise, ala you will need to create a series of symlinks from / usr / local / lib | include in / usr / lib | include or “play around” with LDFLAGS / CPPFLAGS.
Putting mod_admin_extra to expand the list of available commands in ejabberdctl and via xmlrpc:
Stage 2:
The main configuration file ejabberd.yml is replete with examples and comments of developers, so it is not difficult to configure it. In yaml-configuration files, tabulation is important, the habr editor ate it for me, so be careful! I will dwell only on some parameters:
In the ejabberd_c2s section:
For external authorization:
To connect to mysql:
Register "admin":
We switch part of the modules to work with mysql:
We connect mod_admin_extra:
We rule ejabberdctl.cfg (parameters are individual, but I cite those that are registered with us):
Go to the mysql server, create a base there and fill it with a dump from mysql.sql from the sql folder of the ejabberd sources.
Run!
Create an admin user:
Stage 3:
To create a cluster, we take another server and perform the procedures described above.
In order for replication between nodes to work correctly (and indeed to assemble a cluster), it is necessary that the nodes can connect to each other on port 4369 and the ports that you specified in FIREWALL_WINDOW.
Copy the /var/lib/ejabberd/.erlang_cookie file from the first node to the second one, and if there are no problems with connections between the nodes, then we assemble the cluster:
Call mnesia: info ().
If everything is good, then in the running db nodes list you will have both nodes.
Change the table storage scheme, exit and run:
Stage 4:
A little about the parameters of the system:
If everything went well, then going to the admin panel of youserver.com:5280/admin/nodes/ in the Running Nodes will be both nodes.
mod_admin_extra significantly expands the list of commands that can be passed via ejabberdctl. If you left the ejabberd_xmlrpc module (which has become embedded in ejabberd since version 13), then you can send commands to the jabber server via XmlRPC. We, for example, manage the roster, groups, vcards and other things through special workers written in php. But, in fact, you can connect to xmlrpc from anything.
As for the consumed resources, then for 100 thousand connected users it is required somewhere under 25-30 Gb of memory (this is when most of the data is stored in mysql, that is, purely for servicing the connected clients). If you store data in mnesia, the figure can easily reach 100 Gb or more, but it all depends on the number of users and data in the database.
Unfortunately, there are very few examples of successful ejabberd installations on the web that work with tens of thousands of connected users. Personally, I only stumbled upon the “rumors” that the significantly patched ejabberd is the basis of the WhatsApp services. And, in itself, erlang is a separate song, to which it is not so easy to find an approach :) Ejabberd mailing lists are also in a half-dead state, and it is problematic to consult with someone about possible high-load problems, therefore, much has to be studied with bitter experience. I am glad that ejabberd is now developing quite actively (compared to the period until the second half of 2013), a new functionality appears. This year there were several major contributors from the community who regularly correct bugs left over from 2.x versions or newly acquired ones.
I will be glad to find "accomplices" who use ejabberd in their services - maybe we can productively share the experience with each other!
PS I will be grateful for the indication of shortcomings (preferably in a personal) and welcome any comments. Thank!
A small preface: the 13th saw the light not so long ago, and the 14th version of ejabberd followed this year. Process One decided to change the versioning scheme for a number of reasons, and version 2.x is now considered obsolete. The new ejabberd shared on two branches - ejabberd Community Server and ejabberd Business Edition and began to develop quite actively.
In our case, FreeBSD was chosen as the system, since it is necessary to maintain a fairly large number of connections per node (~ 100k).
')
I don’t see any reason to stop installing the system, so under the cut we will immediately proceed to the setting.
Some ejabberd data will be stored in mysql, so odbc libraries must be installed on the system. We also use the external authorization script in php, so php is also needed.
Stage 1:
Put the prerequisites (git, fop, libyaml, wget, unixODBC, expat, wx30-gtk2, php5, etc):
pkg install git fop libyaml wget unixODBC-2.3.2 expat wx30-gtk2 php5 ... (+ )cd /usr/ports/java/openjdk7/make install cleanDownloading the latest version of Erlang (at the time of writing of article 17.2.2):
mkdir /usr/local/src && cd /usr/local/srcgit clone git://github.com/erlang/otp.gitWe collect, we put it:
cd /usr/local/src/otpautoconf -f./configure --prefix=/usr/local --enable-kernel-poll --with-ssl --enable-threads --with-odbc --enable-smp-supportgmakegmake installNext, download ejabberd and ejabberd-contrib:
cd /usr/local/srcgit clone git://github.com/processone/ejabberd.gitgit clone git://github.com/gamenet/ejabberd-contrib.gitCreate user ejabberd, specify / var / lib / ejabberd as the home directory.
We compile and install ejabberd:
cd /usr/local/src/ejabberdautoconf -f./configure --enable-mysql=yes --enable-odbc=yes --enable-lager=yes --prefix=/usr/local --localstatedir=/var --enable-user=ejabberdgmake && gmake installWhen building erlang and ejabberd, various nuances may arise, ala you will need to create a series of symlinks from / usr / local / lib | include in / usr / lib | include or “play around” with LDFLAGS / CPPFLAGS.
Putting mod_admin_extra to expand the list of available commands in ejabberdctl and via xmlrpc:
cd /usr/local/src/ejabberd-contrib/mod_admin_extra./build.shcp ./ebin/mod_admin_extra.beam /usr/local/lib/ejabberd/ebin/Stage 2:
The main configuration file ejabberd.yml is replete with examples and comments of developers, so it is not difficult to configure it. In yaml-configuration files, tabulation is important, the habr editor ate it for me, so be careful! I will dwell only on some parameters:
hosts:- "youserver.com"In the ejabberd_c2s section:
backlog: 1024For external authorization:
auth_method: externalextauth_program: "/usr/local/bin/<_>.php"extauth_instances: 12extauth_cache: 72000To connect to mysql:
odbc_type: mysqlodbc_server: "mysql_server_ip"odbc_database: "dbname"odbc_username: "username"odbc_password: "password"odbc_pool_size: 16odbc_keepalive_interval: 600Register "admin":
admin:user:- "admin": "yourserver.com"We switch part of the modules to work with mysql:
mod_last:db_type: odbcmod_muc:db_type: odbcmod_offline:db_type: odbcmod_privacy:db_type: odbcmod_private:db_type: odbcmod_roster:db_type: odbcmod_shared_roster:db_type: odbcmod_vcard:db_type: odbcWe connect mod_admin_extra:
mod_admin_extra: {}We rule ejabberdctl.cfg (parameters are individual, but I cite those that are registered with us):
ERL_MAX_PORTS=524288FIREWALL_WINDOW=4200-4210ERL_PROCESSES=5000000ERL_MAX_ETS_TABLES=262144ERLANG_NODE=ejabberd@node1Go to the mysql server, create a base there and fill it with a dump from mysql.sql from the sql folder of the ejabberd sources.
Run!
su - ejabberdejabberdctl startCreate an admin user:
ejabberdctl register admin yourserver.com passwordStage 3:
To create a cluster, we take another server and perform the procedures described above.
In order for replication between nodes to work correctly (and indeed to assemble a cluster), it is necessary that the nodes can connect to each other on port 4369 and the ports that you specified in FIREWALL_WINDOW.
Copy the /var/lib/ejabberd/.erlang_cookie file from the first node to the second one, and if there are no problems with connections between the nodes, then we assemble the cluster:
su - ejabberderl -sname ejabberd@node2 -mnesia dir '"/var/lib/ejabberd/"' -mnesia extra_db_nodes "['ejabberd@node1']" -s mnesiaCall mnesia: info ().
If everything is good, then in the running db nodes list you will have both nodes.
Change the table storage scheme, exit and run:
mnesia:change_table_copy_type(schema, node(), disc_copies).q().ejabberdctl startStage 4:
A little about the parameters of the system:
loader.conf
kern.ipc.maxsockets=2400000. kern.ipc.nmbclusters=0 net.inet.tcp.reass.maxsegments=2048 vm.pmap.shpgperproc=400 hw.em.rxd=4096 hw.em.txd=4096 hw.em.rx_int_delay=100 hw.em.tx_int_delay=100 hw.em.rx_abs_int_delay=1000 hw.em.tx_abs_int_delay=1000 dev.em.rx_processing_limit=-1 net.inet.tcp.hostcache.hashsize=4096 net.inet.tcp.hostcache.bucketlimit=100 net.inet.tcp.hostcache.cachelimit=65536 net.inet.tcp.syncache.hashsize=4096 net.inet.tcp.syncache.bucketlimit=120 net.inet.tcp.syncache.cachelimit=131072 net.inet.tcp.tcbhashsize=524288 net.isr.defaultqlimit=4096 net.isr.bindthreads=1 net.isr.maxthreads=4 net.link.ifqmaxlen=1024 sysctl.conf
kern.ipc.shm_use_phys=1 kern.ipc.somaxconn=8192 kern.maxfiles=3000000 kern.maxfilesperproc=2700000 kern.maxvnodes=256000 kern.random.sys.harvest.ethernet=0 kern.random.sys.harvest.interrupt=0 kern.sync_on_panic=1 net.inet.icmp.bmcastecho=0 net.inet.icmp.drop_redirect=1 net.inet.icmp.maskrepl=0 net.inet.ip.intr_queue_maxlen=256 net.inet.ip.maxfragpackets=1024 net.inet.ip.portrange.first=1024 net.inet.ip.portrange.last=65535 net.inet.ip.portrange.randomized=0 net.inet.ip.redirect=0 net.inet.ip.sourceroute=0 net.inet.ip.accept_sourceroute=0 net.inet.tcp.blackhole=2 net.inet.tcp.drop_synfin=1 net.inet.tcp.fast_finwait2_recycle=1 net.inet.tcp.finwait2_timeout=3000 net.inet.tcp.hostcache.expire=1200 net.inet.tcp.keepinit=5000 net.inet.tcp.maxtcptw=65536 net.inet.tcp.msl=5000 net.inet.tcp.recvbuf_auto=0 net.inet.tcp.recvspace=65536 net.inet.tcp.sendbuf_auto=0 net.inet.tcp.sendspace=131072 net.inet.tcp.syncookies=1 net.inet.tcp.tso=0 net.inet.udp.blackhole=1 net.inet.udp.recvspace=32768 net.isr.direct=1 net.route.netisr_maxqlen=1024 vfs.ufs.dirhash_maxmem=100000000 If everything went well, then going to the admin panel of youserver.com:5280/admin/nodes/ in the Running Nodes will be both nodes.
mod_admin_extra significantly expands the list of commands that can be passed via ejabberdctl. If you left the ejabberd_xmlrpc module (which has become embedded in ejabberd since version 13), then you can send commands to the jabber server via XmlRPC. We, for example, manage the roster, groups, vcards and other things through special workers written in php. But, in fact, you can connect to xmlrpc from anything.
As for the consumed resources, then for 100 thousand connected users it is required somewhere under 25-30 Gb of memory (this is when most of the data is stored in mysql, that is, purely for servicing the connected clients). If you store data in mnesia, the figure can easily reach 100 Gb or more, but it all depends on the number of users and data in the database.
Unfortunately, there are very few examples of successful ejabberd installations on the web that work with tens of thousands of connected users. Personally, I only stumbled upon the “rumors” that the significantly patched ejabberd is the basis of the WhatsApp services. And, in itself, erlang is a separate song, to which it is not so easy to find an approach :) Ejabberd mailing lists are also in a half-dead state, and it is problematic to consult with someone about possible high-load problems, therefore, much has to be studied with bitter experience. I am glad that ejabberd is now developing quite actively (compared to the period until the second half of 2013), a new functionality appears. This year there were several major contributors from the community who regularly correct bugs left over from 2.x versions or newly acquired ones.
I will be glad to find "accomplices" who use ejabberd in their services - maybe we can productively share the experience with each other!
PS I will be grateful for the indication of shortcomings (preferably in a personal) and welcome any comments. Thank!
Source: https://habr.com/ru/post/236615/
All Articles