January 1, 2020 / Greg / 0 Comments
There’s a few different options out there offering insight into how to create a custom speed test sensor to PRTG, but today I’m going to use this one from Nicolai Pederson as my jumping off point. Nicolai was using an .exe file from Github that hadn’t been updated in sometime now, and when I started messing with it, I noticed that the speed test really didn’t run long enough to give a valid result. Also, Ookla’s Speedtest.net recently released their own CLI tool, so I wanted to take what Nicolai did and make it work with the new took from Ookla, which is actually pretty easy. So, we’re going to follow his instructions, with a few changes to his .bat file and I’m going to make one change to keep the results consistent.
-
- Download the Speedtest.net CLI app from Ookla.
- Copy those files to “C:\Program Files (x86)\PRTG Network Monitor\Custom Sensors\EXEXML”. For sake of simplicity, that’s going to be our working directory.
- Open up a command prompt, cd to “C:\Program Files (x86)\PRTG Network Monitor\Custom Sensors\EXEXML”, and run “speedtest.exe -L”. This is going to give you a list of servers close to you. I would recommend picking the server of your ISP if it’s on the list.
- Once you have your server picked, make note of server ID. We’re going to be using that in our .bat file shortly. In my case, I’m using the Spectrum server, ID 16969.
- Open up Notepad and copy the following. We’re going to create a .bat file with it.
-
@ECHO off
SETLOCAL EnableDelayedExpansion
SET “Latency=”
SET “Download=”
SET “Upload=”
FOR /F “tokens=4,7,8 delims=,” %%A IN (‘”C:\Program Files (x86)\PRTG Network Monitor\Custom Sensors\EXEXML\speedtest.exe” –accept-license -s 16969 -f csv’) DO (
SET Latency=%%~A
SET Download=%%~B
SET Upload=%%~C
)
ECHO ^<PRTG^>
ECHO ^<result^>
ECHO ^<Channel^>Ping Latency^</Channel^>
ECHO ^<value^>%Latency%^</value^>
ECHO ^<Mode^>Absolute^</Mode^>
ECHO ^<Unit^>TimeResponse^</Unit^>
ECHO ^<Float^>1^</Float^>
ECHO ^<ShowChart^>1^</ShowChart^>
ECHO ^<ShowTable^>1^</ShowTable^>
ECHO ^</result^>
ECHO ^<result^>
ECHO ^<Channel^>Download^</Channel^>
ECHO ^<value^>%Download%^</value^>
ECHO ^<Mode^>Absolute^</Mode^>
ECHO ^<volumeSize^>MegaBit^</volumeSize^>
ECHO ^<float^>0^</float^>
ECHO ^<unit^>SpeedNet^</unit^>
ECHO ^<ShowChart^>1^</ShowChart^>
ECHO ^<ShowTable^>1^</ShowTable^>
ECHO ^</result^>
ECHO ^<result^>
ECHO ^<Channel^>Upload^</Channel^>
ECHO ^<value^>%Upload%^</value^>
ECHO ^<Mode^>Absolute^</Mode^>
ECHO ^<volumeSize^>MegaBit^</volumeSize^>
ECHO ^<float^>0^</float^>
ECHO ^<unit^>SpeedNet^</unit^>
ECHO ^<ShowChart^>1^</ShowChart^>
ECHO ^<ShowTable^>1^</ShowTable^>
ECHO ^</result^>
ECHO ^</PRTG^>
- Replace the server 16969 with the server ID of your choice. The reason we’re going to use the same server, preferably from your ISP, is to have consistency with your speed test. If you’re using multiple servers, you could get varying results as you don’t know what kind of bandwidth each server has. And, if you’re using your own ISP, they’re a lot more likely to give you truly accurate results and less likely to block you if you run the test a lot.
- Save the file as something like speedtest.bat in the working directory, “C:\Program Files (x86)\PRTG Network Monitor\Custom Sensors\EXEXML”. Just make sure you remember what you saved it as.
- Go to PRTG and create a new sensor. The sensor type will be “EXE / Script Advanced”, then name it and select your “speedtest.bat” for EXE/Script under Sensor Settings.
- Once you have the sensor created and you gather some data, go in change the scanning interval. You obviously don’t want this thing scanning every 60 seconds or so. I set mine to scan every 6 hours, but you can set yours as you see fit.
So, why did I do this when Nicolai had already done the work? Well, the Github .exe that Nicolai uses only runs for a few seconds and doesn’t really run long enough to give an accurate reading, also it tried to use servers that didn’t exist anymore. If you check his website, you’ll see there’s some comments about people complaining that they were getting incorrect results. The “official” Speedtest CLI app solves that problem. Also, the official app can spit out JSON, but PRTG doesn’t like the format, and I’m not smart enough to know how to parse the data into a format that it does like, so I had to figure out a way to get the data I wanted into a format that PRTG wanted.
Now, for those of you like me that aren’t smart and want to figure out what that .bat file is doing, I’ll explain. The speedtest.exe file is spitting out the data in a CSV format (the “-f csv” behind the command is the formatting). The “FOR /F “tokens=4,7,8 delims=,”” in the .bat is a loop telling it that the output is comma delimited, and you want to look at the data that’s behind the 4th, 7th, and 8th comma. YOU MAY NEED TO CHANGE THIS! The reason It’s set to 4,7,8 on mine is because the output of the actual command comes back with the very first line being “Spectrum – Columbus, OH”, and it reads the comma before OH as, well, a comma. If the output of your command doesn’t have a comma there, you may have to change it. To find out for sure, you can run the following:
speedtest.exe –accept-license -s #YOURSERVERNUMBER# -f csv
The count commas. If you’re not sure what data is where, you can run the following and it will tell you.
speedtest.exe –accept-license -s #YOURSERVERNUMBER# -f csv –output-header
That will tell you what data is in which location. You’ll get something like this:
“server name”,”server id”,”latency”,”jitter”,”packet loss”,”download”,”upload”,”download bytes”,”upload bytes”,”share url”
“Spectrum – Columbus, OH”,”16969″,”7.495″,”0.786″,”N/A”,”110863914″,”4621941″,”1425470568″,”32103608″,”https://www.speedtest.net/result/c/73cf23fa-84cd-4473-a816-4154424fd027″
Of course, now that you know what’s being parsed and how, you can add more data to this if you want, like packet loss, jitter, download bytes, etc. You just need to follow the example set in the .bat file, make sure you test it out. You can run the .bat from the CLI and see the data or check for errors before creating the sensor. Since I first posted this, I’ve gone ahead and created an example that pulls all the information from the CLI output except packet loss into one place. You can download that here, and just rename it to .bat to run it. Don’t forget to change your server ID too! One more note of changes I made in the .bat file between his and mine; I removed his remarks, added a ~ between “%%~A”, etc to remove the quotes from the response in the CSV file, and cleaned up the formatting a bit, and removed the “00” from the upload and download values (they’re not needed). I should also note that I spent over 2 hours trying to figure out why I was seeing good, clean data at the CLI, but only zeros in PRTG. Let’s just say there’s very a reason the “–accept-license” option is set in the command now <grrr….>. Once you’re done, you’ll end up with a working sensor!
Update 3/11/20: As Roman in the comments found out, if your country or area requires that you accept other types of licenses or data protection regulations (like the GDPR in the EU), you may need to feed that into the command. It took me 2 hours to realize I needed to feed the “–accept-license” option, and it took Roman 3 days to figure out he needed to feed the “–accept-gdpr” option. Whenever you first run the command from the CLI, you will be asked to accept certain things, like the license and possibly the GDPR and anything else. REMEMBER WHAT IT IS YOU ACCEPT. PRTG is going to run this command as a different user, which is why you have to feed the “–accept-license” option to the command; just because you accepted the license doesn’t mean PRTG did. If you’re getting zero’s on your sensor, try to figure out what other options need to be accepted in your area when you issue the command. Then go into the comments below and thank Roman for chasing this down over 3 days so you didn’t have to.
January 15, 2016 / Greg / 12 Comments
Now the fun starts! All the parts arrived and it was time to put them on the test bench to burn things in. This is my first ever dual processor build, so it was definitely a learning experience. Nothing is really different, it’s just twice as much. When the motherboard arrived, it was absolutely beautiful.
I couldn’t wait to get everything put together and get it on the bench. The RAM had arrived a few days earlier and I knew the processors were supposed to be arriving via USPS later that day. I was antsy with anticipation! Then the letter carrier arrived and my CPUs were ready to go into the board.
Now that I’ve got the processors and RAM installed, lets put on the coolers, get it mounted to the bench, and get it wired up.
Thank God I sprung for the EATX version of this test bench. For those of you that are curious, this is the Highspeed PC Half-Deck Tech Station XL-ATX and it’s a great little test bench. There’s no metal parts to come in contact with the motherboard, so no worries about shorting things out.
Now that everything was together, it was time for the smoke test. In case you didn’t know, computers actually run on smoke. If the smoke escapes, it stops working, and the first POST of a new computer, especially one with an open-box motherboard and CPUs from eBay, is the time when that smoke is most likely to escape. Luckily, this one passed the smoke test.
I slapped the 6x 6TB drives into a carrier and put the LSI 9211-8i HBA in to start burning everything in. I added a USB fan to keep the HBA cool since there wasn’t any airflow on that side of the board. The HDD rack has it’s own fan. Getting the HBA flashed to the IT firmware was quite the pain, but I’ll save that for it’s own post.
First up was to run memtest86 and check the 128GB of ECC DDR3. I ran this for a few days to really beat up the memory, as memtest86 runs over and over and over again until you stop it. After the first pass, I knew I was going to be good because there were no errors found, but I let it run for a while just to be safe. I love this picture because it shows 32 CPUs found and 16 started (It’s 16 physical cores with hyper-threading for a total of 32)!
After burning in the machine for a while, it was time to transplant it into its permanent home, the Rosewill rackmount chassis. Problem was, there was already a computer in there.
So, I pulled out the old motherboard (which will actually end up being my new gaming rig) to have a fresh case to start with.
After moving some standoffs around, the motherboard fit in perfectly.
My original plan was to use the onboard SAS ports for the six 3TB drives and use the LSI HBA for the six 6TB drives, then use the onboard SATA3 ports for the two SSDs. I ended up using all 8 onboard SAS ports instead. FreeNAS doesn’t care what controller the drives are plugged into. I’m not sure if that was a good idea or not, and I plan on looking into it more. If it turns out it is a bad idea, I’ll just move all the 6TB drives to the same controller.
Once everything was put together, it was time to boot it up in the chassis for the first time. I hit the power button and… nothing. The fans spun up for a second, then the whole thing shut down. I had no idea what was going on. The first thing that came to mind was the fact that I couldn’t find the second CPU power cable for the EVGA power supply, so I “borrowed” one from a Corsair PSU I had. I went ahead and unplugged all the drives to see if maybe something there was shorted and it wasn’t. I grabbed the Corsair PSU and plugged it into the second CPU and the computer booted. Ok, maybe it was the cable…
I pulled the EVGA PSU out, put the Corsair PSU in, kinda redid all the cable management, and hit the power button…
Nothing. WTF??? This thing was working fine on the test bench! I did a little more troubleshooting and figured that if it was working fine on the test bench, I’d just go grab that PSU and use it. Out with the Corsair PSU, in with a Rosewill 1000W that I use for the test bench. I hit the power button and… IT’S ALIVE!
The drives are all recognized, FreeNAS boots up without a problem, and we’re good to go. My wife actually did the cable management in the chassis because I was fed up with dealing with it. I was originally going to start with a fresh install of FreeNAS, but since it booted up with no issues, I decided to just stick with the current install, though I found out pretty quick that I needed to delete all the tunables created by autotune as they didn’t update to the new hardware. My ARC was still limited to 12GB.
The box has been up and running damn good for over a week now, minus a few reboots with me doing stuff.
I built the new volume with the six 6TB drives and started moving some stuff to that new pool.
So, that’s the hardware build of my new FreeNAS server. Next, we’ll get into the software part of the whole thing. Even though I already have FreeNAS installed and running on this machine, I’ll run through the install procedure using another box and we’ll get into the meat and potatoes of getting FreeNAS, Plex, and all the Plex Automation setup.
December 22, 2015 / Greg / 0 Comments
I think that, in this day and age, everyone should have a NAS at their house. For those of you that don’t know what I’m talking about, NAS stands for ‘Network Attached Storage’. A NAS is handy for storing all sorts of things, primarily backups of your computers and your media. In my case, I have a lot of movies and TV shows for my various media players. I also have a ton of photos and videos from over the years, as well as from my drones. Having a large NAS means that I don’t have delete anything. My NAS also acts as a server for various other things that I’ll get into in another post.
For your NAS to be effective, it needs to have lots of space and have enough room to expand. You also need to have an effective operating system running the NAS. For this build, I’m going to use FreeNAS. I had been planning to build this thing for a while, but didn’t get around to finally getting everything setup and running until July 31, 2015. Since then it’s been running pretty stable, but I used an Intel G3220 and 8GB of RAM when I first put it together and I’ve outgrown that processor and RAM, so it’s time for an upgrade. Here’s the hardware list of everything that’s going into the machine:
- Intel Core i7-4790K CPU
- ASRock Z97 EXTREME6 ATX LGA1150 Motherboard
- G.Skill Ripjaws X Series 32GB (4 x 8GB) DDR3-1600 Memory
- 6x WD Red 3TB 3.5″ 5400RPM HDD
- Rosewill RSV-L4412 - 4U Rackmount Server Chassis, 12 SATA / SAS Hot-swap Drives
- EVGA SuperNOVA 1000G2 1000W 80+ Gold Certified Fully-Modular ATX Power Supply
The only thing that’s carrying over from the previous build are the 6 WD Red 3TB hard drives and the actual FreeNAS install. I was going to just upgrade the CPU and the RAM, but some pins got bent on the ASUS Z87-A motherboard I had, so it needed to get upgraded too. I also figured that while I was at it, I’d put it in a nice rackmount chassis too.
The build went rather smooth. I pulled the hardware out of the old mid-tower case and moved it into the rackmount chassis. I had originally planned on using some M.2 SSDs for boot drives, but ran into some issues. First, the drives I bought weren’t compatible with the Ultra M.2 slot on the motherboard. Secondly, the other M.2 slot ate two of my SATA ports on the motherboard. Because I didn’t bother to read the manual, it took me quite a while to figure out why those two drives weren’t being seen by the BIOS. Ultimately, I got everything put together and all 6 drives were being recognized. FreeNAS booted right up without any issues. I’ll probably pick up an Ultra M.2 SSD in the future to use as L2ARC since it’s so freaking FAST.
More info will be posted soon on how I’m going to automate my media collection and sharing.
August 3, 2010 / Greg / 1 Comment
In my last post I taught you how to forward a port on the ASA 5505 running version 8.3 from the CLI. Some of you prefer to use the ASDM to do you changes, so I guess I’ll show you how to do it from there. The ASDM is a bit of a learning curve for someone that’s used to the CLI, and most CLI guys hate a GUI with a great passion. I can go either way. I use the ASDM to make some changes simply because I want to learn it and there’s some guys coming into the field today that were taught on the GUI rather than a command line.
In this lesson I’m using ASDM version 6.3(1) and ASA version 8.3(1). Since we added a web server in the last post, let’s make this one an FTP server. The FTP server’s IP is the same as the web server, 10.9.8.7/24 and we’re running over the standard FTP port, 21.
First off, we want to start up the ASDM and connect to the ASA. Once there, click on the button at the top of the screen, then the button near the bottom left, and finally select near the top left. You’ll now be at a screen that looks something like this:
Click for larger version
Now we need to create a new object, so click on “Add” under Addresses, then “Network Object”.
Now we need to fill out our new window. Once you fill out the name, IP address and description, you need to drop down the NAT box and fill it out. Click the “Add Automatic Address Translation Rules” box, leave the type as “static” and set the translated address as the outside interface.
We now need to go to the Advanced menu from the Add Network Object window and setup the port forwarding. The source will be inside, destination is outside. Protocol in this instance is TCP and our port is 21, both real and mapped.
Click “OK” twice and your object will be created as well as the port forward. Now we just need to add the access rule. On the left side of the screen, just above the NAT Rules is your Access Rules. From there we want to click “Add” and “Access Rule”.
We need to create the rule on the outside interface, coming from any IP to the FTPServer using FTP as the service.
Once you click OK, your rule is added. You don’t have to add a description like I did in the image above this one, I just did that for the hell of it. When you click “Apply” at the bottom of the screen, the ASDM will issue the commands to the ASA. I have preview turned on, so I can always see what commands are being sent to the device before they are actually sent. If you followed all the steps above and you have preview turned on, you’ll see the following:
And you’ll notice that those are the exact 4 commands that I gave in the last post about doing it from the CLI! Now you can forward any port you want from either the CLI or the ASDM!
On a side note, I know a lot of guys hate the ASDM. When I was writing this post and going through all of this I was kinda upset when I saw that I had 10 pictures for 4 lines of code. The good thing about the ASDM is that you have everything right there at your disposal and you really don’t need to know the vernacular of IOS. The drawback is that it will take you longer to get things done at first, but once you get used to it, it can be just as fast.
August 3, 2010 / Greg / 1 Comment
So it’s been a month and a half since I posted an update, and it’s 4:15 am right now. I can’t sleep and I found out there’s another networking blog out there using the same WP theme as me, so I figured I better put something up here since it was fresh in my mind. Well, now that the niceties are out of the way, let’s get to work.
I recently added an ASA 5505 to my home network at the edge. Obviously, when I did, all of my port forwards went to hell because the ASA is now blocking everything. I run a web server on one of my servers here and I like to be able to access it because I keep a lot of tech manuals and other stuff on there. Well, I went about trying to set up port forwarding the old way and learned real quick that this pops up when I do:
ERROR: This syntax of nat command has been deprecated.
Please refer to “help nat” command for more details.
Yeah, that sucks. On the new version of the ASA OS, global has gone the way of the dodo. I did a bunch of searches on Google to figure it out and everything I ran across was very hard to decipher. That’s why I’m writing this. You can setup a port forward in 4 quick and easy steps. Just change the things that are underlined to fit your network and you’ll be just fine.
In this example, we want to be able to access a web server behind the firewall. We’ll assume you are using the standard HTTP port, the web server’s internal IP address is 10.9.8.7/24, and that you at least know what you’re doing enough to be configuring an ASA in the first place. I’ll give you the steps, then I’ll explain.
Step 1: Create a new object group for you web server.
asa5505(config)# object network Webserver
Step 2: Add the IP of the web server to the network group.
asa5505(config-network-object)# host 10.9.8.7
Step 3: Forward the port via the NAT command.
asa5505(config-network-object)# nat (inside,outside) static interface service tcp www www
Step 4: Exit back to the root and add the access list
asa5505(config)# access-list outside_access_in permit tcp any object Webserver eq www
That’s it! Now, let’s explain what’s going on here. Cisco has started moving more and more towards use of object groups in their configs. It makes things easier, especially when you have a situation where you have 20 web servers behind the firewall and you want to add 1 more in. Rather than having to rewrite a whole bunch of ACL’s, you just add the IP of the new web server into the object group and everything is done for you. After you create the object group (in this instance a network object, you can also create service objects), you add the IP of the specific object (or objects) that you want to point to. So here our web server is 10.9.8.7. If you want to send port 80 to more than 1 IP on your internal network, just add more IP’s to that object group.
Now is the fun part. While we’re in the object group, we need to NAT port 80 only to that specific object group, hence you’re still at “asa5505(config-network-object)#” prompt. Now type “end” to get back to the regular config terminal and we need to open that port in the ACL. Yes, www = 80. You can type either one and you get the same result. If I have to go through and explain NAT, how it works and why I enter in that specific command to forward the port, then there’s a possibility that I’d need to send you an invoice for my time because we would be here for a while.
This works for ANY port forward. If you want to RDP into a machine, simply replace port 80 (all those www’s you see up there) with 3389. There is one caveat. You can only do one port forward per object group. So let’s say that our web server is also an FTP server and you want port 21 to forward as well as port 80. You’re going to have to create a whole new object group (object network FTPServer), put the same IP in the group (host 10.9.8.7), do the nat command again (nat (inside,outside) static interface service tcp ftp ftp), exit back to the root of config, and add the access list (access-list outside_access_in permit tcp any object FTPServer eq ftp).
This should get you up and running with you port forwards in no time flat. It is a bit of a pain in the ass to have to create a new object group for every port you want to forward, and maybe there’s someone out there that’s reading this right now thinking “dude, you don’t have to create more than one group! You can just do…”. Well, you need to enlighten the world with this knowledge and post it in the comments section. And if you’re too scared to do so, shoot me an email to greg(at)gregledet(dot)net.
I’d also like to thank Stefan Fouant for an excellent class today on JUNOS Switching. I learned a lot in his class and you can learn a lot from his website. Check it out and tell him Greg sent ya!