Install syslog-ng on CentOS 6.3


syslog-ng is an open source syslog server that can be installed on many *nix servers. In my example, I installed it on a 64bit CentOS 6.3 server with minimal package.

1) Download and extract the source code.

wget http://www.balabit.com/downloads/files?path=/syslog-ng/open-source-edition/3.4.0alpha3/source/eventlog_0.2.12.tar.gz

wget http://www.balabit.com/downloads/files?path=/syslog-ng/open-source-edition/3.4.0alpha3/source/syslog-ng_3.4.0alpha3.tar.gz

tar xvzf eventlog-0.2.12.tar.gz

tar xvzf syslog-ng-3.4.0alpha3.tar.gz

2) Install dependencies.

yum -y install kernel-headers* glibc-headers-* glibc-devel-* gcc-* zlib-* zlib-devel-* libffi-* libffi-devel-* e2fsprogs-devel-* keyutils-libs-devel-* libselinux-devel-* libsepol-devel-* libselinux-devel-* krb5-devel-* openssl-devel-* pcre-devel xz

wget ftp://ftp.gtk.org/pub/glib/2.32/glib-2.32.1.tar.xz
tar xvJf glib-2.32.1.tar.xz
cd glib-2.32.1
./configure
make
make install

3) Install eventlog.

cd eventlog-0.2.12
./configure
make
make install

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH

4) Install syslog-ng.

cd syslog-ng-3.4.0alpha3
./configure
make
make install

5) Create the syslog-ng configuration file.

cd /usr/local/etc

mv syslog-ng.conf syslog-ng.conf.bak

mkdir /var/log/syslog-ng

vi syslog-ng.conf

@version:3.4
@include “scl.conf”

####################################
##Source
source remote {
udp(ip(0.0.0.0) port(514));
tcp(ip(0.0.0.0) port(514));
};

####################################
##Destination
destination host01 {
file(“/var/log/syslog-ng/host-01.log”);
};

destination host02 {
file(“/var/log/syslog-ng/host-02.log”);
};

destination host03 {
file(“/var/log/syslog-ng/host-03.log”);
};

####################################
##Filter
filter host01 {
host(“host-01_server_name“);
};

filter host02 {
host(“host-02_server_name“);
};

filter vmh03 {
host(“host-03_server_name“);
};

####################################
##Log
log {
source(remote);
filter(host01);
destination(host01);
};

log {
source(remote);
filter(host02);
destination(host02);
};

log {
source(remote);
filter(host03);
destination(host03);
};

6) Check the /var/log/syslog-ng folder. The remote host’s syslog files should be stored there now.

7) The open source edition syslog-ng does not come with a web UI, so you have to install it separately. I used a simple tool named logstash.

wget http://semicomplete.com/files/logstash/logstash-1.1.1-monolithic.jar

8) Create the logstash configuration file (logstash-syslog.conf) in the same folder as where the jar file is.

input {
file {
type => “linux-syslog”
path => [ "/var/log/syslog-ng/*.log" ]
}
}

output {
stdout { }
elasticsearch { embedded => true }
}

9) Run logstash.

java -jar logstash-1.1.1-monolithic.jar agent -f logstash-syslog.conf — web –backend elasticsearch:///?local

10) Go to http://<server_IP>:9292, and use the Lucene’s string query language to search.

image

About these ads

48 comments on “Install syslog-ng on CentOS 6.3

  1. Apparently no it can’t….

    Machine#tar xvJf glib-2.32.1.tar.xz
    tar (child): xz: Cannot exec: No such file or directory
    tar (child): Error is not recoverable: exiting now
    tar: Child returned status 2
    tar: Error is not recoverable: exiting now

    • -rw-r–r– 1 root root 6138200 Oct 8 21:05 glib-2.32.1.tar.xz
      The file is there but I can’t get it to engage and break it out.

  2. Really nice write up. I could see that my syslog server is receiving logs through tcpdump but not in /var/log/syslog-ng folder. I even went and created a file that correspond to the /var/log/syslog-ng/host-02.log but still does not write to a any log files. I did some research and see that others have their syslog-ng.conf in /etc/syslog-ng/ folder; whereas yours is in /usr/local/etc. I assume this doesn’t affect the install. I also tried to restarting the rsyslog service but still no content in my log file. Below is out my sysconfig file. Any suggestion is greatly appreciated. Thanks!

    @version:3.4
    @include “scl.conf”

    ####################################
    ##Source
    source remote {
    udp(ip(0.0.0.0) port(514));
    tcp(ip(0.0.0.0) port(514));
    };

    ####################################
    ##Destination
    destination pacore1 {
    file(“/var/log/syslog-ng/core.log”);
    };

    ####################################
    ##Filter
    filter pacore1 {
    host(“192.168.1.1“);
    };

    ####################################
    ##Log
    log {
    source(192.168.1.1);
    filter(core);
    destination(core);
    };

  3. It should not work as you used the wrong destination name and filter name in your log sections. The correct one should be:

    ##Log
    log {
    source(remote);
    filter(pacore1);
    destination(pacore1);
    };

  4. I am having problems with this command: java -jar logstash-1.1.1-monolithic.jar agent -f logstash-syslog.conf — web –backend elasticsearch://10.1.15.80/?local

    I get this:
    “LogStash::Runner” org.jruby.exceptions.RaiseException: (InvalidOption) invalid option: -backend

    If I run: java -jar logstash-1.1.1-monolithic.jar agent -f logstash-syslog.conf

    It runs fine but netstat shows nothing listening on 9292

  5. Hi Jackie,
    when i run the logstash i get the below error.

    Exception in thread “LogStash::Runner” org.jruby.exceptions.RaiseException: (InvalidOption) invalid option: –web

      • I did put double dashes in front of both web and backend but i still got the error, below is the full output:
        Exception in thread “LogStash::Runner” org.jruby.exceptions.RaiseException: (Inv alidOption) invalid option: –web
        at OptionParser.complete(file:/home/kunle/logstash-1.1.1-monolithic.jar! /META-INF/jruby.home/lib/ruby/1.9/optparse.rb:1459)
        at org.jruby.RubyKernel.catch(org/jruby/RubyKernel.java:1197)
        at OptionParser.complete(file:/home/kunle/logstash-1.1.1-monolithic.jar! /META-INF/jruby.home/lib/ruby/1.9/optparse.rb:1457)
        at OptionParser.parse_in_order(file:/home/kunle/logstash-1.1.1-monolithi c.jar!/META-INF/jruby.home/lib/ruby/1.9/optparse.rb:1271)
        at org.jruby.RubyKernel.catch(org/jruby/RubyKernel.java:1197)
        at OptionParser.parse_in_order(file:/home/kunle/logstash-1.1.1-monolithi c.jar!/META-INF/jruby.home/lib/ruby/1.9/optparse.rb:1264)
        at OptionParser.order!(file:/home/kunle/logstash-1.1.1-monolithic.jar!/M ETA-INF/jruby.home/lib/ruby/1.9/optparse.rb:1258)
        at OptionParser.permute!(file:/home/kunle/logstash-1.1.1-monolithic.jar! /META-INF/jruby.home/lib/ruby/1.9/optparse.rb:1349)
        at OptionParser.parse!(file:/home/kunle/logstash-1.1.1-monolithic.jar!/M ETA-INF/jruby.home/lib/ruby/1.9/optparse.rb:1370)
        at OptionParser.parse(file:/home/kunle/logstash-1.1.1-monolithic.jar!/ME TA-INF/jruby.home/lib/ruby/1.9/optparse.rb:1360)
        at Agent.parse_options(file:/home/kunle/logstash-1.1.1-monolithic.jar!/l ogstash/agent.rb:192)
        at Agent.run(file:/home/kunle/logstash-1.1.1-monolithic.jar!/logstash/ag ent.rb:303)
        at logstash.runner.run(logstash/runner.rb:70)
        at org.jruby.RubyProc.call(org/jruby/RubyProc.java:258)
        at logstash.runner.run(logstash/runner.rb:95)
        at logstash.runner.main(logstash/runner.rb:37)
        at logstash.runner.(root)(logstash/runner.rb:120)

      • I don’t know why it does not work for you, it looks to me still has something to do with the double dashes thing. Did you just copy and paste the command line? If so try to type them in.

  6. **A little new to this**
    After I wget both syslog-ng and eventlog, I try to extract the files and I get the error message: tar (child): syslog-ng-3.4.0alpha3.tar.gz: cannot open: no such file or directory. I get the same error message for eventlog. What am I not doing correctly?

  7. I managed to install syslog-ng 3.2 on CentOS. What I need is to install new version with GeoIP support enabled. I am new to Linux and would appreciate if you could let me if this version has GeoIP support, and if not how can I add to it.

  8. Thanks. When I try to install the dependencies I get the following:

    yum -y install kernel-headers* glibc-headers-* glibc-devel-* gcc-* zlib-* zlib-devel-* libffi-* libffi-devel-* e2fsprogs-devel-* keyutils-libs-devel-* libselinux-devel-* libsepol-devel-* libselinux-devel-* krb5-devel-* openssl-devel-* pcre-devel xz
    Loaded plugins: fastestmirror, protect-packages, refresh-packagekit, security
    Loading mirror speeds from cached hostfile
    * atomic: www6.atomicorp.com
    * epel: mirror.steadfast.net
    * rpmforge: mirror.us.leaseweb.net
    Setting up Install Process
    No package glibc-headers-* available.
    No package glibc-devel-* available.
    No package gcc-* available.
    No package zlib-devel-* available.
    No package libffi-devel-* available.
    No package e2fsprogs-devel-* available.
    No package keyutils-libs-devel-* available.
    No package libselinux-devel-* available.
    No package libsepol-devel-* available.
    No package libselinux-devel-* available.
    No package krb5-devel-* available.
    No package openssl-devel-* available.
    No package pcre-devel available.
    Nothing to do

    What I am doing wrong.

    • You did not do anything wrong. What it means is that those packages are not available in your yum repositories.

      Go to ‘/etc/yum.repos.d/’, and check what are there. The default CentOS yum repositories are: CentOS-Base.repo CentOS-Debuginfo.repo CentOS-Media.repo CentOS-Vault.repo

  9. Hi

    Following is what I have in my /etc/yum.repos.d/. How to do I add the other repositories? Thanks in advance for your guidance

    -rw-r–r– 1 root root 119 Jul 27 2011 afni-local-rhel-base.repo
    -rw-r–r– 1 root root 957 Nov 4 21:52 epel.repo
    -rw-r–r– 1 root root 1056 Nov 4 21:52 epel-testing.repo

    • It seems someone has changed the yum repository in your server. If you want to add more repo sources, firstly find your server version then either copy the repo files from other server running on the same version or download and install the repo rpm packages .

  10. Thanks. I got the repos. I have another newbee question. To install GeoIP support, I will change the step four of your instructions as follows – right?

    4) Install syslog-ng.

    cd syslog-ng-3.4.0alpha3
    ./configure
    make
    make install –include enable-geoip

  11. I’m using syslog-ng-3.3.8 and eventlog-0.2.12. I’ve installed all of the dependencies and when I do ./configure –enable-dynamic-linking for syslog-ng I get the following error at the end of the run

    configure: error: Package requirements (eventlog >= 0.2.12) were not met: No package ‘eventlog’ found Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix. Alternatively, you may set the environment variables EVTLOG_CFLAGS and EVTLOG_LIBS to avoid the need to call pkg-config

    Any help here would be appreciated.

  12. I am having some trouble getting the modules/afsocket to compile. Getting the following error:

    In file included from /usr/include/net/ethernet.h:27,
    from /usr/include/libnet.h:97,
    from afinet-dest.c:43:
    /usr/include/linux/if_ether.h:110: error: expected specifier-qualifier-list before ‘__be16′
    make[4]: *** [libafsocket_notls_la-afinet-dest.lo] Error 1
    make[4]: Leaving directory `/usr/local/src/mongo/syslog-ng-3.4.1/modules/afsocket’
    make[3]: *** [all] Error 2
    make[3]: Leaving directory `/usr/local/src/mongo/syslog-ng-3.4.1/modules/afsocket’
    make[2]: *** [all-recursive] Error 1
    make[2]: Leaving directory `/usr/local/src/mongo/syslog-ng-3.4.1/modules’
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory `/usr/local/src/mongo/syslog-ng-3.4.1′
    make: *** [all] Error 2

    This occurs when it is compiling the afinet-dest.c in modules/afsocket.

    Any ideas ?

  13. I am trying to get multiple hosts to log into a single file as they are part of a cluster and any one of the host should be able to send a syslog message to this server. Is what I have here viable? I didn’t seem to work?

    @version:3.4
    @include “scl.conf”

    ####################################
    ##Source
    source remote {
    udp(ip(0.0.0.0) port(514));
    tcp(ip(0.0.0.0) port(514));
    };

    ####################################
    ##Destination
    destination test{
    file(“/var/log/syslog-ng/test);
    };
    ####################################
    ##Filter
    filter host01 {
    host(“10.244.146.161“);
    };

    filter host02 {
    host(“10.244.146.162“);
    };

    filter host03 {
    host(“10.244.146.163“);
    };

    ####################################
    ##Log
    log {
    source(remote);
    filter(host01,host2,host3);
    destination(test);
    };

    TIA

  14. hi Jackie,

    following up with your config, I got smoothly in the installation but then I checked under /var/log/syslog-ng, NO file was stored there.
    Next I did “netstats -na | grep 514″ and see that the port is not up yet. Do we need to reboot the machine after installed?
    I could see the message came from my remote host but somehow it got unreachable on this server.
    05:43:24.086849 IP6 (hlim 64, next-header ICMPv6 (58) payload length: 167) 2a01:d0:890e:8000::4 > 2a01:d0:890e:20::5: [icmp6 sum ok] ICMP6, destination unreachable, length 167, unreachable port, 2a01:d0:890e:8000::4 udp port 514
    05:43:24.086869 IP6 (hlim 63, next-header UDP (17) payload length: 98) 2a01:d0:890e:20::5.514 > 2a01:d0:890e:8000::4.514: [udp sum ok] SYSLOG, length: 90

  15. Hi, i have problems with local loogin.. I´m trying with

    source local { unix-stream(“/dev/log”); internal(); udp();};
    destination host01 {file(“/var/log/auth.log”);};
    filter f_auth { facility(auth, authpriv);};
    log {source(local);filter(f_auth);destination(host01);};

    I want to loggin local authentication only. When i start the daemon it never create the auth.log file..

    Help me please..

    Thanks

  16. Here again, i’m having trouble with the instruction ‘make’ to syslog..alfa. The following message appears: “make *** : no targets specified and no makefile found. Stop”

      • No, actually I had to update yum with “yum update yum” command, and after that ‘yum update’ to update all dependecies, then I started the process all over again and I got it right. Thats a good tip for those who can’t run some command. But I’m having another trouble here.

        @version:3.4
        @include “scl.conf”

        ####################################
        ##Source
        source remote {
        udp(ip(0.0.0.0) port(514));
        tcp(ip(0.0.0.0) port(514));
        };

        ####################################
        ##Destination
        destination nocserver {
        file(“/var/log/syslog-ng/nocserver.log”);
        };

        ####################################
        ##Filter
        filter nocserver {
        host(“nocserver.trt20.intra”); // in here does it have to be my ip address or not?
        };

        ####################################
        ##Log
        log {
        source(remote); // in here does it have to be “remote” or my ip address?
        filter(nocserver);
        destination(nocserver);
        };

        When I configure my syslog-ng.conf, my log file does not appear. Is there something wrong?
        Thanks for helping anyway

  17. Hi Jackie….

    I am getting following error while compiling syslog-ng-3.4.0alpha3. I am a rookie… please help me out

    checking for LIBDBI… no
    checking for dbi_initialize in -ldbi… no
    checking whether to enable SQL support… no
    checking for GLIB… no
    configure: error: Package requirements (glib-2.0 >= 2.10.1 gmodule-2.0 gthread-2.0) were not met:

    No package ‘glib-2.0′ found
    No package ‘gmodule-2.0′ found
    No package ‘gthread-2.0′ found

    Consider adjusting the PKG_CONFIG_PATH environment variable if you
    installed software in a non-standard prefix.

    Alternatively, you may set the environment variables GLIB_CFLAGS
    and GLIB_LIBS to avoid the need to call pkg-config.
    See the pkg-config man page for more details.
    [root@sf-log-nms syslog-ng-3.4.0alpha3]#

  18. I have some problems running the comand ” java -jar logstash-1.1.1-monolithic.jar agent -f logstash-syslog.conf — web –backend elasticsearch:///?local ” have this ans [root@localhost 29112013]# java -jar logstash-1.1.1-monolithic.jar agent -f logstash-syslog.conf —web –backend elasticsearch:///?local
    Aborted (core dumped)
    pls need some help

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s