Kyh's blog

Kyh's random blog of stuff

Some Internet Security Basics

This is a quick guide on the things you should be doing for say, your families or non-technical friends machines. It won’t protect them from everything, but will hopefully result in less calls to you! (Or if you’re not a techie, then less calls to your more techy friends!)

Everything i’ve listed here is free (or was at the time of writing). There are more steps you can take which either require more effort on the users part (Such as javascript/application whitelisting, etc) or things you have to pay for, but that’s for another day. Note that a few things are Windows 7+ specific.

Our threat model also doesn’t include nation states or “APTs”. Nor will it 100% protect you against someone that is targeting YOU specifically. (But it will make it harder) Just stuff that normal users probably have to worry about without sacrificing usability too much.

The two biggest sources of malware these days are a) Phishing and b) Not Patching Your Stuff (Or a combo of both), so let’s start with

Patching:

Turn on those auto-updates. Just do it. I know they’re annoying and that Microsoft STILL hasn’t sorted out their shit (Which they promised us would happen by the time Vista was out) but you have to patch. Just do it. Having your machine reboot at 3am in the morning (Or in 15 minutes) and possibly losing work is nothing compared to having your precious data ransomed via Cryptolocker. Patch all the things. Patch them Now.

Web Browser: Get rid of the IE (and Edge in windows 10. It doesn’t currently support extensions/addons) icon and replace it with Chrome or Firefox. Since Chrome comes bundled with flash (that autoupdates) make sure it’s on “Click to run:” http://www.howtogeek.com/188059/how-to-enable-click-to-play-plugins-in-every-web-browser/

For both browsers you want: ublock Origin (ads), Privacy badger (ads + trackers) and HTTPS Everywhere:

Chrome: Privacy Badger ublock Origin HTTPS Everywhere

Firefox: HTTPS Everywhere ublock Origin Privacy Badger

As a fall back, you can use DNS/hosts based blocking, if you’re comfortable enough following the instructions at: https://github.com/StevenBlack/hosts (Or in addition to adblocking in browser extensions)

Passwords:

Ah, our age old enemy. Passwords. Bets are most people without a password manager re-use their passwords across sites, or have a “base” password with the sites name tacked on it. This is bad because when one site gets hacked, attackers will take all the cracked passwords and try them across all the other different sites and if you reuse passwords, they’ll then have control of your accounts. (And you won’t!)

Despite Lastpass being bought out by LogMeIn i’d still suggest LastPass https://lastpass.com for use as an easy to use/available everywhere password manager. Have it autogenerate and auto-complete password fields, because this is Just Easier. (OS X does have it’s own built in keychain, but there seems to be issues with syncing with Chrome/FF in the latest version of OS X. YMMV)

At the very least, try to get the rolling code (TOTP) 2FA enabled on your email account that is used for password resets and whatever Cloud Backup solution you decide to go with, as well as whatever password manager you go with. (Lastpass does 2FA so does Google drive, Dropbox etc). Most decent 2FA implementations include backup codes, incase you lose access to your phone. Write these down and store them in a safe place. You can also print out the QR codes you get given to scan, just make sure you store these offline in a safe place. (A fireproof safe, your wallet, etc)

Ideally, enable 2FA on everything that gives you the option to. But feel free to click the “Trust this device” when logging into services with 2FA. This makes it a lot more user friendly.

For storing/using 2FA codes you can either go with the default Google Authenticator or Authy: https://www.authy.com/ (Which works across multiple devices and stores your 2FA codes in the Cloud. Just make sure you password protect the Authy backups)

AntiVirus:

Just go ahead and leave Windows Defender turned on. It’s a decent AV and while it probably won’t catch 0days or brand new-never-seen-before stuff, it will catch most of the stuff that most users will see. Plus, it’s free! (A bunch of the Free AV vendors have started monitising their “free” AV products by injecting ads or selling your browser history: http://www.wired.co.uk/news/archive/2015-09/17/avg-privacy-policy-browser-search-data )

Anti-Malware:

Install Malwares Bytes Anti-Exploit (Free): https://www.malwarebytes.org/antiexploit/ which will stop most (not all!) “0day” or drive by attacks on browsers. The free version only protects browsers and Java however.

Also install Malwarebytes Anti-Malware https://www.malwarebytes.org/antimalware/ and set it up to run background scans every day (Which it should do by default)

Backups:

If you have an android or iphone, make sure you enable the backup services provided. iCloud can backup your photos etc. On Android with a google account, you can do the same. In both cases, if you don’t have a large datacap, make sure you find the setting that only allows backups over wifi. Also make sure you enable 2FA for these services. You can also use Google Drive for storing backups of files, as well as something like Dropbox. Both offer file revisions (Up to 30 days only, however!) so if you get cryptolockered, you can in theory go back and recover the uncryptolockered versions. Make sure you keep copies of important documents in GD/Dropbox/etc and keep them updated. Also make sure that you have a local copy! Cloud storage can go away and keeping your only copy “in the cloud” is a bad idea.

Full Disk Encryption:

If you have a version of Windows that supports it, turn on Bitlocker Full Disk Encryption. Most windows laptops these days have a TPM so you don’t need a separate password for Bitlocker. Make sure you store a copy of the recovery key in your Microsoft account as well as somewhere physically separated from your machine. (Either in a safe or at friends place). Somewhere that a burglar isn’t going to be get to if they break into your house/car and steal your machine. This is particularly important for laptops.

Lock Screen:

A pretty basic one here. Set a screensaver for however long you think you’ll need (5-15 minutes is a good option) and then have it require a password to unlock. I’ve gotten into the habit of always locking my screen when I get up from the keyboard (Windows-L in windows land). Anyone with the right hardware and a minute or two can completely compromise your machine if you leave it unlocked.

That’s it for this post. There are more things you can do to protect yourself (and others) but require more effort and/or cost. If your Threat Model includes more advanced bad actors than what this post entails, you might want to ignore some of the advice here (Such as anything that stores anything in the cloud) but for most people, the above is probably the most you can do for minimal effort.

Why You Shouldn’t Be Doing Password Expiry

I’m sure everyone has at least one account (most likely a work account) where the password expires every days and you have to IMMEDIATELY change your password on logging in, before you can do any more work.
Often this is every 60-90 days and really, who has the will or time to come up with an actually secure password in the face of being locked out of your account? You just want to get your work done, right? So you just increment the number or perhaps letters (Months in the form of Jan, Feb, Mar are good candidates. ) So how’d this come about? Somehow it got baked into a bunch of compliance bullshit, as we’re about to see.
From what I can find, this came from the DoD in the 1970’s, where they figured they should make passwords expire before their mainframes could crack the (DES?) hashes. There are also a few papers from the 1970’s that reference this. https://www.computer.org/csdl/proceedings/afips/1977/5085/00/50850027.pdf http://csrc.nist.gov/publications/history/ande72.pdf

Compliance:
ISO27K2:

9.2.3 Management of privileged access rights
h) for generic administration user IDs, the confidentiality of secret authentication information should be maintained when shared (e.g. changing passwords frequently and as soon as possible when a privileged user leaves or changes job, communicating them among privileged users with appropriate mechanisms)
This only applies to shared accounts, which you shouldn’t have. All accounts, even service accounts should be named/audited accounts. If you do have them, they should be changed every single time an employee who has had access to them leaves the company. (This never happens in most companies)
9.4.3 Password management system
A password management system should:
;enforce regular password changes and as needed;
Note the AS NEEDED. This means that upon suspicion or likelihood of compromise a users password will be needed. Note it doesn’t specify a time frame, therefore a regular “1 year” password expiry is sufficient.
ISO27K does NOT mandate forced 90 day password expiry.

HIPAA:

I couldn’t find any specific text that says you must specifically expire passwords every (x) days. If anyone knows of this, let me know.

PCI-DSS:

8.2.4 Change user passwords/passphrases at least once every 90 days.
This says: Passwords/phrases that are valid for a long time without a change provide malicious individuals with more time to work on breaking the password/phrase
However, 90 (45) days is plenty for an attacker to crack even strong hashes if I user chooses a weak password as well as plenty of time to get backdoor access that doesn’t depend on legitimate credentials.

8.2.5 Do not allow an individual to submit a new password/phrase that is the same as any of the last four passwords/phrases he or she has used.
This requires a reversible encrypted storage of the previous four passwords. If an attack gets their hands on this..
Users will also just find a way around this. A good example is “Cr4pPasswordFeb” then “Cr4pPasswordApr”. This will get around most “incrementing not allowed” rules.

The above is mitigated by: 8.3 Incorporate two-factor authentication for remote network access originating from outside the network by personnel (including users and administrators) and all third parties, (including vendor access for support or maintenance).

NOTE: PCI-DSS policy only applies to accounts that view/access cardholder data or access systems with cardholder data. Normal desktop user/admin accounts are NOT required to follow 8.2.4/8.2.5. PCI-DSS is all about scope.

Hitrust:

Control Reference: 01.d User Password Management: “vii.” passwords shall be changed at least every 90 days or based on the number of accesses; “viii.” passwords for privileged accounts shall be changed at least every 60 days;” However, you can get a “get out of jail for a bit longer” card here You can use “alternative controls”, where an example given is proposing the extension of password expiration to one year by increasing complexity of the password.

Of course, this brings us to the fact that nowhere does anyone actually explain what 60-90 day password expiry is meant to protect against. Assumedly the reason is to limit the windows of exposure.

The problems with (short) password expiry:

Users will use not very good passwords because they will have to just forget it 60/90 days later.

Why bother trying to create and memorise a good strong password/passphrase if the system just makes you change it in 2-3 months time? Screw that! You’ll come up with the shortest most memorable password you can think of in order to get your work done. Then next time it expires, just increment a number or add an a or something on the end.
One common technique is just jamming a month on the end, then incrementing that. Most “You can’t use your last four passwords” or “No incrementing!” systems will be fooled by that.

Doesn’t protect against poor hashes in regards to offline password cracking.

If you’re talking about AD hashes here, then as far as I can tell AD hashes are just a single round MD5 (or MD4). My video card (An R9 290) can do ~8 BILLION MD5 hashes a SECOND. If an attacker gets your AD hashes, they will blow through those way before most of the passwords expire.
If your password hashing algorithm isn’t bcrypt/scrypt/PBKDF2 or Argon2 then anyone with your password hashes is still going to be able to crack your users passwords before their passwords expire. Even if they don’t, you still have the incrementing problem. Even with bcrypt/etc an attacker with a good dictionary is likely to get a few of them, your password complexity rules be damned.
If they’re really determined, they can just throw more and more hardware at the problem and crack them faster than your users can change them.

Increase in IT help desk load/cost. Decreased productivity. Raises the odds of successful social engineering attacks against the IT help desk.

This is a people problem, where if people actually create decent (hard to remember!) passwords, they’re more likely to forget them than ones that don’t expire or ones they can take time to get right. (Especially if the password change happens on a Friday!)
This increases the load/cost on the IT help desk due to higher amounts of password resets. Decreases (and pisses off) user’s productivity. Also increases the odds of a successful social engineering attack.
Know when someone started at (employer)? Or maybe just get lucky? Call up the help desk, pretend to be a poor frustrated user who really just needs to get their work done, but they’ve forgotten their (newly set) password. IT will more than likely let you do it.
There is also the problem of saved and automated credentials. If you have an app using your credentials for something (subversion or an automated script) then as soon as your user changes their password, the app with the now invalid credentials will trip your account lockout, locking the user out of their account and more load on the IT helpdesk.

SSO vs non SSO accounts.

If you use things like Skype, Hipchat, Join.me and other services that aren’t SSOed, you have a few options. (None of which are good).
You can try to force users to rotate their 3rd party app/website logins the same time they do their main account. Then scapegoat them when a) they just set all those passwords to the same thing or b) they forget to rotate the 3rd party accounts and then one of those 3rd party services gets hacked and oh hey, the attacker now has access to all the other 3rd party services you use if a) or b) Well, they violated policy (and still get fired) but at least the attacker didn’t get their domain creds!
You can try and force them to all be different, but good luck with that. (See the Increase load on help desk when your users forget them all)
Your only good option here is to ONLY use sites/apps that you can federate with your SSO. Anything else will suck for your users. (Of course, if your SSO creds get leaked, you have no not-compromised way to contact your users)

Attackers (if they’re any good) will find a way to establish a backdoor into your system that doesn’t rely on the stolen/cracked credentials.

This could be the attacker setting up their own account on your system, that they will just change the password on if you have a password expiry set.
Otherwise they will establish some kind of backdoor that can call out to systems they control or some other kind of persistence that doesn’t rely on the stolen credentials that you are dutifully rotating every 90 days.
Some attackers may wait a certain time before using the credentials or using them to establish a foothold inside your infrastructure, this will usually happen before passwords expire. If you’re being targeted specifically, they can do this quickly. (Inside of a day or two).
Other attackers will do the damage as soon as they get in.

Users will write down their new (complex but no longer secure) passwords.

Especially if they need access to that resource NOW. What happens to those old post notes? Well, they’ll probably stick around.
You can even have a password policy that says they’re not allowed to do this, but hey, they’re only human. But that’s OK, you can always scapegoat them. (As i’ve heard from a few C levels. Yeah, really.)

Some attacks don’t rely on stealing usernames/passwords.

This includes stuff like phishing, exploits used against software vulnerabilities (Whether they’re 0days or not), pass the hash, social engineering, physical attacks, badly designed network/cryptographic protocols, bad HTTPS, no credentials on API/etcs, default creds. Basically all of the OWASP top 10. None of these necessarily rely on valid credentials, so your 90 day password resets won’t protect you against these.

Persistent keyloggers/Remote Access Trojan’s will just steal the new passwords.

Someone drops a keylogger/RAT on your users box either by drive by’s/malvertising or phishing and if your endpoint management system or IDS/DPI boxes don’t pick that up then they’ll just deliver your users newly changed credentials up to the attackers. You’ve just forced your user to change their password for nothing!

Password expiry does not address password disclosure.

So some how you find out that your users password got leaked or disclosed? You get them to change it! A forced 90 day password expiry does nothing to address this. Infact you’ve just pissed off the user some more, since they’ll just have to change it after 90 days again. (Or the next disclosure, whichever comes next)

2FA mitigates the risk/vulnerability? “addressed” by password expiry.

Use 2FA. Seriously, just roll it out everywhere you can. The newer compliance stuff requires 2FA anyway. Congratulations, you’ve just addressed the issue password expiry supposedly addresses! Now stop forcing your users to change their passwords every 90 days. You’ve already inconvenienced them with 2FA. (Whether T/HOTP, SMS, U2F, etc).
If an attacker gets your user’s credentials and you have 2FA the attacker will go to use them and hopefully trigger your monitoring system. (You have monitoring right?)
Even though it’s possible to steal 2FA token secrets, you’re still in a way better place than you were without 2FA + password expiry.
Even better, a TOTP changes every 30-60 seconds. Tell that damned auditor that your users login credentials change every 30 seconds.
Do you also force SSH keypairs to be rotated every 90 days? Or just the password protecting them? (Which if you haven’t updated your ssh client will be hashed with single round MD5. Yes, really. You want bcrypt formatted SSH keys

Shared accounts.

Now, everyone (should) know that you should never use shared accounts. Infact, a bunch of the compliance bullshit requires that you don’t have them. But we all know everyone has service or shared accounts. Does your password expiration policy apply to these too?
If so, you’ve now increased the chance of disclosure or malicious use of these credentials, since you have to communicate the new passwords to all the users that use these accounts.
Not to mention if the new password isn’t communicated to everyone, you’ve just locked out those users.

Logging/monitoring clutter.

You have logging and monitor those logs with alerting/dashboards/SIEM/etcetc right? Well all those password changes and password resets are going to clutter up your logs and ping your monitoring systems.
Hope you don’t miss an attacker resetting or other messing with accounts because you thought it was just normal password expiry activity!

Phishing.

Mentioned above, but phishing is a major attack vector and so many users still fall for it. (This isn’t necessarily their fault, email is not and has never been a secure/trustworthy communication medium)
Password expiry won’t protect your users against phishing as the attacker can either drop a RAT on their machine or have systems setup to immediately login with the stolen creds and do whatever they’re going to do.
Even if they don’t use them immediately, the attackers still have plenty of time to use those credentials and establish persistance.

Bad JML (Joiners/Movers/Leavers) processes.

I suppose this is an argument that works for password expiry. When users leave if your JML process is so bad that you don’t immediately expire/lockout/disable their account then they or an attacker can only get into your network/infrastructure until their password expires.
The fix here is to make sure your JML process isn’t bad! Have a document process in place for the return of all your hardware and disable all their accounts by the next day. You should setup alerting for this where possible.
As an alternative, just set an automatic “lock out account of 90 days of inactivity”. This slightly increases your window of exposure, but only by up to 90 days.

What you should be doing instead:

2FA.
Logging and some kind of monitoring/alerting and user behaviour analysis. An Appliance or tools on their aren’t good enough, you’ll need some good people to watch/use the tools!
Some kind of decent password manager system rolled out company wide.
Setup an internal password strength checker like zxcvbn and gamify setting a strong password.
Don’t scapegoat your users when they get phished or disclose passwords. Doing so discourages them from coming forward when they screw up or something goes wrong and you won’t find out until it’s too late.
Change passwords/credentials if you suspect or have confirmed any kind of breach involving credentials. However, make sure you’ve confirmed that you’ve cleaned up the avenue of attack/credential leakage before you do this.
Sign up for a domain wide (Or just your own personal address) alert for dumps involving your companies email addresses at Have I been pwned Users will use their work email addresses to sign up for crap they have to download as part of their jobs. (The adobe dump is a good example!)

Thanks to @TCFoxtaur, #kiwicon and a few other peeps on the twitters.

Further reading:

https://www.computer.org/csdl/proceedings/afips/1977/5085/00/50850027.pdf 1977. Obviously a whole lot of this no longer applies.
http://csrc.nist.gov/publications/history/ande72.pdf from 1972.
https://www.schneier.com/blog/archives/2010/11/changing_passwo.html https://docs.google.com/viewer?url=http://research.microsoft.com/en-us/um/people/cormac/papers/2009/SoLongAndNoThanks.pdf http://cs.unc.edu/~fabian/papers/PasswordExpire.pdf https://www.cs.auckland.ac.nz/~pgut001/pubs/book.pdf (password lifetimes chapter, page 572+)
https://www.cl.cam.ac.uk/~rja14/shb10/angela2.pdf http://healthitsecurity.com/2012/12/04/cio-addresses-password-change-frequency-security-innovation/ http://community.hcca-info.org/hcca/communities/community-home/digestviewer/viewthread?GroupId=121&MID=25599 http://csrc.nist.gov/publications/drafts/800-118/draft-sp800-118.pdf https://www.sans.org/reading-room/whitepapers/basics/password-security-thirty-five-years-35592 https://www.cerias.purdue.edu/site/blog/post/password-change-myths

Patching Openssh6.7p1 With U2F Support

U2F is (yet another) 2/multi factor standard. Hopefully this time this one will stick. Yubikey have a nice explanation of how this works

Specifications are available at http://fidoalliance.org/specifications/download/ Google has added support for the U2F keys already and now @zekjur has a patch against OpenSSH6.7p1 to add support for U2F.

There’s a bug entry up at https://bugzilla.mindrot.org/show_bug.cgi?id=2319 for this. NOTE: This code hasn’t been peer reviewed AFAIK and i’m not sure about how they’re using the appid/origin part of the U2F specs here.

Unfortunately there seems to be an issue with U2F AND PAM support, so we’ll build OpenSSH without PAM support and run it along side the current install.

So here’s how we build this sucker on a debian/ubuntu based box: First we need some build dependancies. The ubuntu packages you’ll need are: libtool gtk-doc-tools pkg-config libjson0-dev libhidapi-hidraw0 libjson0-dev libhidapi-dev autoconf zlib1g-dev gengetopt help2man libssl-dev build-essential

We’ll need libu2f-host, which will give us the U2F USB host support we need on the client side. Obviously we’ll need the client to support this as well, so you’ll have to build this locally as well. For win64 users, have some cygwin64 built binaries (Although, the key registration seems to be broken with this binary, whoops)

Grab the master branch of libu2f-host from: https://github.com/Yubico/libu2f-host and build with make ; sudo make install

Next grab the openssh6.7p1 source (Don’t forget to verify the source..) and apply the patch attached to the bug at https://bugzilla.mindrot.org/show_bug.cgi?id=2319 (You want the diff/raw unifed links..) Copy the patch to the openssh-6.7p1 directory and patch with patch -p1 <patchfilename. It should patch without any errors.

After patching, run rm configure to get rid of the pre-u2f patch configure file and then run autoconf -i to rebuild it. Run the configure script with ./configure --with-u2f (and with any other options you want/need, remembering that with the current patch, PAM may be broken) Run make ; sudo make install. By default this will install to /usr/local/ so be careful if you’re already running your sshd from there.

If we’re not replacing our normal sshd, edit /usr/local/etc/ssh/sshd_config and change the Port option to whatever port you’d like. Also add: U2FAuthentication yes AuthenticationMethods password,u2f to the sshd_config file. (Replacing “password” with “publickey,password” or whatever you currently use. This will still look for publickeys in ~/.ssh/authorized_keys but it WON’T use any PAM auth methods) Tweak the config however else you’d like.

Fire up the new sshd with /usr/local/sbin/sshd, then on the client side plug in your U2F key (after you’ve built the client with U2F support, as above) and run ssh -o U2FMode=registration my.server.example -p newport > /tmp/u2f-key.pub

Login with your usual credentials and touch the U2F key when prompted. It will error out with a “Permission denied” and dump the registration info into /tmp/u2f-key.pub. The contents of this file need to go into the users ~/.ssh/authorized_keys file on the server.

Once you’ve done then, you should be able to ssh to the new sshd port with your password/publickey and it will ask you to touch your U2F security key when appropriate. Do so and it should log you in. Yay!

Cyanogenmods Updater Vulnerable to MITM Attack

So it turns out that Cyanogemod’s built in updater gets it’s update information by contacting http://download.cyanogemod.org/api

Note the http:// part there. It also turns out there’s no signature verification of the flashable .zip file that the custom recovery uses to update. What this means is that anyone who can MITM your connection (Which means the NSA to anyone who can manipulate the BGP routing table all the way down to anyone who can own your router/has access to your local LAN/WLAN) can change where the cyanogenmod update looks for the image file that CM will flash.

The updater connects to download.cyanogenmod.org on port 80 and asks:

1
{"method":"get_all_builds","params":{"device":"hammerhead","channels":["stable","snapshot","RC","nightly"]}}

In reply the server usually sends:

1
2
3
4
5
6
7
8
9
10
11
12
13
{
 "id": null,
 "result": [
  {
   "url": "http://get.cm/get/jenkins/59291/cm-11-20140216-NIGHTLY-hammerhead.zip",
   "timestamp": "1392551863",
   "md5sum": "1318e463181519622aa5bb10be0bd5d2",
   "filename": "cm-11-20140216-NIGHTLY-hammerhead.zip",
   "channel": "nightly",
   "changes": "http://get.cm/get/jenkins/59291/CHANGES.txt",
   "api_level": 19
  },
etc

To MITM someone’s update all you need to do is firstly grab a legit copy of the CM update they’re likely to upgrade to, unzip it, replace whatever you want inside it, md5sum it, then serve it up via http. You replace the URL above with your evil URL, the md5sum with your new md5sum and the filename with your evil file. Then use something like mitmproxy + (Wifi pineapple/ARP spoofing/DNS spoofing/etc) to MITM the targets connection. mitmproxy will let you script/regex requests so you can replace the URL/md5sums automagically so then you end up with something like this:

In this case I just modified the build number of the legit CM update, hosted on a webserver running on 192.168.1.83 and MITMed the CM updater to look there for the new update. The middle image shows a screenshot from a Nexus 7 that succesfully applied the hacked update.

This worked because while CM verifies the md5sum (that we replaced) most custom recoveries either just verify that signatures are valid but not authentic or don’t verify that signatures are valid at all.

I’ve opened a bug with CM, CYAN-3502. Lets hope they can fix this. (Probably just by using HTTPS/SSL and verifying certs!)

Setting Up Splunk With Remote Forwarders

Splunk is a fancy log monitoring application. It lets you pull in all kinds of data from logs and various other inputs, then provide you with a fancy web GUI to generate graphs, reports and alerts.

The free license allows you to process 500MB of data a day, which is fine for a few machines.

Splunk supply linux tarballs/deb/rpm packages, so just go ahead and grab the latest version from the splunk download page and install using your package manager. By default splunk listens on port 8000 on http, which means we’re passing our login in the clear, this is bad, so lets go ahead and enable SSL before we login to it.

You need to start splunk so that it generates a bunch of configs/keys automagically, then edit /opt/splunk/etc/system/default/web.conf and change enableSplunkWebSSL to “true”

You can either go ahead and replace the key/cert paths to your own SSL certs or use the newly generated ones. In this case, if we want to use the one that splunk generated for us, we can check the fingerprint by running openssl x509 -fingerprint -in /opt/splunk/etc/auth/splunkweb/cert.pem and verifying the self-signed cert fingerprint matches.

Login with the default login and change your password to something secure.

Splunk provide a headless version of splunk called splunkforwarder that will ship off logs to your main splunk instance (called receivers), so after installing that on our “client” box, we need to tell it where to ship said logs off to as well as enabled “receiving” on our receiver.

In the webgui, you can do this under Manger->Forwarding and Receiving->Configure receiving-> Add new. Splunk can either listen on a single port for all forwarders or you can setup a port per forwarder. I’d recommend firewalling the receiving ports to only allow access from the forwarder hosts and/or use something like openvpn, since splunk listens on 0.0.0.0 by default.

Since we don’t want our logs going over the internets as plaintext we need to change the input type. Edit splunk/etc/apps/search/local/inputs.conf and change [splunktcp://3333] with [splunktcp-ssl:3333] and then add

1
2
3
[SSL]
serverCert=$SPLUNK_HOME/etc/auth/server.pem
rootCA=$SPLUNK_HOME/etc/auth/cacert.pem

If you have reverse DNS setup for the forwarding host, you can also change connection_host to “dns” so it’ll use that instead of the IP address.

At this point we need to configure our forwarder to use a data input, in this case i’m just going to use all of /var/log/.

Edit

Secure TLS Cipher-algos for Your Webserver.

Perfect Forward Secrecy: Attacks: BEAST (CBC, patched TLS1.1/1.2), CRIME (TLS compression, requires MITM + injectioning javascript via http/etc), BREACH (requires HTTP compression, reflect user input in http response bodies + CSRF token in http response headers), RC4 biases, Truncation attack (injecting unenrypted TCP FINs), Lucky13 (CBC padding-timing oracle attack against MAC) Version rollbacks to insecure ciphers + false start. Renegotiation attack.. hhrgrgh firefox doesn’t support TLS1.1/1.2, so if you drop RC4, you’re vulnerable to BEAST (if unpatched) CHOOSE.

TLS MACs and then ENCRYPTS.. Due to CBC padding out to blocks, you can fiddle with the cipher text and see what comes out.. https://www.ssllabs.com/ssltest/analyze.html?d=kyhwana.org

Benchmark DHE vs ECDHE.

SSHFP DNS Records

So it turns out there’s several RFCs for adding SSH fingerprints to DNS records. They would be RFCs 4255 and 6594

What is this useful for?

Well, if you’re sshing into a box from a new location and don’t remember your SSH fingerprints (Was it the RSA/DSA/ECDSA one!?), your SSHFP record(s) will let you verify them. This also prevents MITM attacks upon first connections by verifying “out of band” over DNS. Of course, someone can come along and also MITM your DNS lookups, but that’s what DNSSEC is for.

Openssh seems to have support for SSHFP records since forever (I couldn’t find what version) via the VerifyHostKeyDNS option. Versions of openssh 5.7p1+ have support for ECDSA keys, but only added RFC6594 support (ECDSA and SHA256 hashed fingerprints) in 6.1p1. So if you have ECDSA keys, you’ll need to upgrade to 6.1p1 if you want the ssh client to verify the ECDSA fingerprints. (Type 3)

In order to have openssh verify the SSHFP records, you need to use ssh -o verifyhostkeydns=yes or add the VerifyHostKeyDNS option to your .ssh/config.

Adding SSHFP records are as simple as doing a “` ssh-keygen -r hostname ““ on the machine you want to generate the records for and copy/pasting the output into your zone file.

You’ll get output like:

1
2
3
4
5
6
hostname IN SSHFP 1 1 f14c4f33571ffbf90c9d5f15129cc9c041059f41
hostname IN SSHFP 1 2 3cf2edc60ab732743b085b6f069933e2a173c7c8e3bff5f8670a36a9deb3c4df
hostname IN SSHFP 2 1 b3a4503cae6a4d6735628e4e1039b4fc792ecb2c
hostname IN SSHFP 2 2 2e82993087a7cd7f4ed848ce3f189bc306ff60b152c73a8b50b08fe9256d4dcb
hostname IN SSHFP 3 1 d5a2e79a08bf51455a1d693e53742359eb3d4e01
hostname IN SSHFP 3 2 6196ea891b455633bbb0f81f60f4c9b0d55fa5bc8d5eec11d6d6d1e6caf954d9

The first 3 columns being self-explanatory. The fourth column is the type of key (1 = RSA, 2 = DSA, 3 = ECDSA), fifth column is 1 = SHA1, 2 = SHA256 and the last column being the SHA1 or SHA256 hash of the public key. (un-based64ed, as it’s stored in ssh_hostkey.pub base64 encoded)

You can check your SSHFP records by doing a dig +short SSHFP example.org

Now, annoyingly openssh doesn’t do any kind of DNSSEC validation on its own, so it’s only checking the AD (authenticated data) bit on the response from the DNS resolver. Unfortunatley if you’re not running a local or remote validating resolver that you trust the network path to, an attacker can still change/strip the AD bit from the response between you and the resolver.

They can do this because it’s the resolver that verifies the DNSSEC records and simply returns a bit saying “Yes, this record is signed and valid”.

Another hurdle is that doing DNSSEC lookups almost always results in DNS packets that are larger than 512 bytes. (If you run a firewall and you block TCP DNS or UDP DNS packets larger than 512 bytes, you’re an asshole. Stop it).

This is where EDNS-0 (Extension mechanisms for DNS) comes in. You have to set your resolver options (RES_OPTIONS) to explicitly enable edns0 support. In Linux, this is “export RES_OPTIONS=edns0”.

What if you don’t have a validating resolver? Well, you could install unbound + dnssec-trigger but that’s a pain in the arse, so lets use a trick @isomer told me. We’re going to use googles public DNS resolvers, since they validate DNSSEC records now.

1
2
3
4
5
sudo unshare --mount bash
echo "nameserver 8.8.8.8" >newresolv.conf
mount --bind newresolv.conf /etc/resolv.conf
export RES_OPTIONS=edns0
ssh -v  -p 1000 -o VerifyHostKeyDNS=yes user@yourdomain.tld

(You can use test at kyhwana.org if you’d like to see some actual valid SSHFP records.)

What the above does is run a bash shell with the mount namespace “unshared” from the rest of the system, so we can go ahead and bind newresolv.conf with the google DNS resolver IPs to /etc/resolv.conf, this only affects the bash shell we’ve started. We then tell our resolver to use EDNS0 and then ssh to the host with verbose mode turned on, where we should see: debug1: found 6 secure fingerprints in DNS debug1: matching host key fingerprint found in DNS indicating that it’s found the SSHFP fingerprints and that they’re signed correctly!

Huzzah, you can now securely connect to a host that you’ve connected to before and know that you’re not being MITMed! (Assuming our DNS packets aren’t being interferred with between us and googles DNS)