Builds Cliffnotes

From CDZwiki
Jump to: navigation, search

Network Appliances

Most consumer level router/firewall combos are efficient, low-power units intended for home or small office use. These offer rudimentary options for NAT/PAT and other firewall functions, but otherwise tend to be "weak sauce" beyond basic functionality.

For heftier, more complicated work, an enterprise level router/firewall is required. These, however, can be very expensive thus a common solution is to build one's own. This also gives you the ability to specify a system to your liking or needs. In our example, we wish to focus on futureproofing & power use.

Aside from cost, the main difference between these home-brew devices and purpose built enterprise appliances from companies like Cisco is an occasional bulkiness (since an old PC can be used as the base) and a lack of tech support, unless you're willing to pay the developer of the router OS you've selected.

Operating Systems

Home-brew router/firewalls are typically found using two different OSes:

Based in BSD, capable and hearty: pfSense is only limited by the hardware you give it. It's known for its stability, plug-in library, and ease-of-use. pfSense is an excellent OS choice to support any size organization. It has built-in features like a VPN server, DDNS server, full VLAN capability and so on. pfSense is free via the "Community Edition." You can pay for professional support, though there is a large free community out there. If you want a purpose built system, you'll need basic hardware knowledge, else there are pre-built pfSense systems available. Installation is not much different from any other operating system and is well-guided.

Originally created as a firmware mod for Linksys routers, DD-WRT has since reached out to other devices thanks to a talented community of developers and engineers helping the DD-WRT project along. DD-WRT is best for the home or small office setting where a typical consumer level router/firewall wont do, but that doesn't mean DD-WRT can't flex. DD-WRT is free, support is dependent upon the community but is well maintained. Installation of DD-WRT is basically hacking the device - it's akin to rooting a phone, thus some routers may require a ginger touch or specific ritual to function; you may wish to have experience with such mechanics prior to giving DD-WRT a go (there is a possibility of "bricking" your new router).

In our example build, we will go for the reliability of pfSense, and instead of dumping it out on an old system, we will build a custom system for a router-on-a-stick setup.

Hardware

Asrock n3150m, an example of a small board with a low-power CPU built in.

pfSense can be installed on nearly any old computer you have hanging around, though you'll want at least two NICs to achieve a Router-on-a-Stick setup. Since pfSense is so flexible with x86 hardware, you can purpose build a system with inexpensive hardware and get exactly what you're looking for.

My personal suggestion is to select an all-in-one motherboard with a low-power CPU already built in - these often can be found with dual NICs and PCI(e) slots so you can expand via your available ports. The lower-power CPU will also often mean a power-efficient system in the end, thus a cheaper system to run over-all (and better on our environment).

Motherboard & CPU

For a router/firewall appliance, your motherboard research should be deep - the motherboard will be the most important part! Form factors for an appliance should be Micro ATX or ITX. A combo board with built in CPU will save you a lot of time and money, as will a board with dual NICs. Speaking of, make sure those NICs are reliable with the version of pfSense you intend to use (more on *that* later).

If going with a combo board, I would suggest you check the manual of any potential board to see bus speeds and bandwidth if you plan on adding in any cards.

CPUs don't need to be power-houses for these boxes: in fact we want the opposite. Checking the CPU manufacturer's website for chip specs should reveal the power demand - see if you can find something light. As for speeds, 1GHz or higher with two cores is sufficient. Intel Atoms and Celerons are a reliable example.

Vendor-wise, the motherboard is the heart of your appliance, so this is not the place to skimp on cost. Go with a vendor you (and others) trust, something popular with a good warranty. In my own experiences, Gigabyte has great support and products that last, plus they tend to make the little motherboard/CPU combos we're looking for.

Gatekeeper will rely on a Gigabyte made board based on the Intel Celeron 847.

Memory & Storage

G.Skill memory is inexpensive, reliable & has a great warranty.
SanDisk makes a decent mid-range SSD for less, great for appliances.

Appliances are easy when it comes to memory and storage, especially now that prices for both have come down (sans DDR4, which is still stupid).

For memory, DDR3 is more than enough for a router/firewall appliance and will save a lot of money in the end. Depending on the motherboard you've chosen, you may be looking at standard DIMMs or laptop memory. Laptop memory tends to be a little more expensive for the good stuff and sometimes harder to get. The good news is capacity doesn't need to be high: 2GB is plenty for a router/firewall. Speed doesn't need to excel, either; whatever makes sense price-wise will do.

As for vendors, this depends on who the appliance is for. Don't care if it there's downtime? Buy budget memory from a warehouse vendor, maybe it'll keep kickin', maybe it'll burn out. Else, invest in memory with a good warranty; the "good" stuff will still be inexpensive as long as it's not flashy. For our example build, we're going with 2GB of G.Skill DDR3 PC3-10700, split between two sticks for dual channel. May as well, right?

Primary storage for appliances should be SSD based these days; cramped, warm conditions and low-power PSUs are to be expected, but SSDs can excel here. Plus, you rarely need much storage for a primary disk for a dedicated appliance, so you can take advantage of the lower capacity, lower cost SSDs.

Vendors for storage don't vary as much these days, frankly most HDDs all come out of the same, few factories. That said, go with the ones you know that have good warranties: Western Digital, for example, is one I recommend only for their warranty support. Drives die - unless you wanna throw money away, anticipate this "feature" and get a company who'll bail you out when your array is about to crumble.

A SanDisk Ultra 256GB will be just fine for Gatekeeper.

Chassis, PSU & Other Diddies

There exists many an ITX chassis with nice little PSUs built in.
Intel NICs are known for their reliability and performance.

Your chassis should fit the size and intent of your project. For an appliance, we want small. If you've got with MicroATX, you'll have a plethora of small desktop/set top options. For ITX, your case can get really small for a truly purpose built system. Build quality should be "good enough," I don't like flimsy bits, do you?

Though to be completely honest, if this is just for your own use and you're really on a budget, you can just use the box the damn thing came in and get busy with some extra cardboard and tape.

A reliable vendor for projects like this would be StarTech. Make sure candidate chassis have enough ports and room for all the bits you wish to add.

When building an appliance, you're really going to want to hunt down something with a built in PSU, again, saving time and money. It also saves you the headache of making everything fit in a tiny case when the PSU is already designed to be there.

Else, if you're selecting your own PSU, triple check the form factor and see if you can find high-res pictures of the specific model if it's an odd size or shape - sometimes things just don't line up with the smaller cases.

Cheaper PSUs these days are a lot more reliable than they were 10 years ago, but efficiency will still cost you. Even so, I suggest going with the most efficient - look for 80 Plus. As for wattage requirements, a firewall that goes over 100watts would be as concerning as it would be amusing. StarTech can bail you out here once again.

If you've got the room, you should also look to taking advantage of your appliances available bus ports. For a router/firewall, this mostly means more NICs. Intel NICs are often cited as an example of reliability, though are a little costly. The good news is that other manufacturers make NICs with the same Intel chips for less!

Gatekeeper will handle the chassis and PSU together with a StarTech ITX case, which comes with a 200watt PSU with a neat 4-pin molex adapter on the back. It's cramped, but the SSD keeps airflow decent enough and doesn't generate much heat itself. Low-power CPUs tend not to generate much heat, either; the whole appliance balancing thing is all coming together!

Servers

Operating Systems

A server is most often found using two different OSes:

Transparent, flexible, powerful. If you can only spare your time learning a single OS, let it be this one and you shall be free. Linux varies a bit from distribution to distribution, but the overall "feel" and operating base layer is the same. Desktop Linux distros often come with multiple different UI shells for the user to pick from, but a server-based Linux OS will have you mostly interacting with BASH; get used to text-based interfaces and you can whip up anything you need in a matter of minutes, since Linux-based OSes are typically free.

As far Linux alternatives go, and depending on what you intend on doing with your server, Microsoft's Windows Server editions are a great option, despite the new Microsoft update-schema. Traditional Windows Server uses a UI not unlike desktop editions of Windows: the shell is familiar and any Windows "power user" can get around and set things up with ease. Later editions of Windows Server took a few notes from Linux and have started to support the minimal UI route: MS Command Prompt & PowerShell replace BASH for a Linux-like "Windows Server Core" experience. Windows Server.. is not cheap. But you can often play around with trial versions for a few months or get deep discounts for your own copy.

Hardware

Memory & Storage

ECC memory is typically required for servers.
Inland makes a great, inexpensive SSD.

First thing's first: when it comes to a server, not all memory is equal (or even an option). Most "real" servers are going to need ECC and/or Registered memory. Again, checking the manual for your selected system to see what it prefers is going to pay off in this sense.

Without getting too technical, ECC stands for Error-Correcting Code. This means the memory is able to make adjustments and correct common cases of data corruption on the fly. This makes the memory more reliable and, in a sense, faster. Registered memory has a buffer cache built-into each DIMM; again, the idea here is reliability and speed.

When it comes to vendors, the best depends on who & what the server is for. Don't care if it there's downtime? Buy budget memory from a warehouse vendor, maybe it'll keep kickin', maybe it'll burn out. Else, invest in memory with a good warranty; the "good" stuff will still be inexpensive as long as it's not flashy.

If you're on a budget, many retailers will also sell refurbish/rebranded memory for older systems or servers in "bulk" (16GB kits or more).

Primary storage for a server should be SSD based these days; fast read and write times are expected of any system intended to serve the masses, so an SSD will be optimal. We can also take advantage of mechanical HDDs for storage of data that doesn't need to be quite as zippy.

Vendors for storage don't vary as much these days, frankly most HDDs all come out of the same, few factories. That said, go with the ones you know that have good warranties: Western Digital, for example, is one I recommend only for their warranty support. Drives die - unless you wanna throw money away, anticipate this "feature" and get a company who'll bail you out when your array is about to crumble.

If you've gone with a pre-built server as a starting point, you'll most likely have several options for disk configurations with a RAID controller in the front, or at least some supporting card in the back (unless it's been stripped).

For Butter, and for the sake of my power bill, we're going with a cheap Inland SSD. 240GB is plenty for any server OS, and we've still got a lot of room for future disks if we decide to pack on the data.

Chassis, PSU & Other Diddies

The "Pedestal" or Tower format, less common for servers.
Server PSUs are often proprietary & oddly shaped.
An easy way to get around 3.5" drive bays for SSDs.

Your chassis should fit the size and intent of your project. For a server, we want it to conform either to the intent or the environment, meaning we must design it for a rack or the hardware within. For a thrifted system, we're likely to encounter everything already in the appropriate chassis, so look for the style that suits your project best: a two-post, rack-mount chassis is not a bad way to go for smaller, lighter systems. For heftier hosts, a "tower" or "pedestal" design is preferred that's got enough room for proper cooling.

If you decide you want to change the chassis your thrifted system came with, a reliable vendor for projects like this would be StarTech. Make sure candidate chassis have enough ports and room for all the bits you wish to add.

If your server already has a PSU, great! You saved yourself a headache. Else, if you're selecting your own PSU, triple check the form factor and see if you can find high-res pictures of the specific model if it's an odd size or shape - sometimes things just don't line up with the smaller cases. When it comes to pre-built servers, PSUs are often proprietary and purpose-built, be sure they're available at a price affordable to you if you plan on investing in such a system.

Cheaper PSUs these days are a lot more reliable than they were 10 years ago, but efficiency will still cost you. Even so, I suggest going with the most efficient - look for 80 Plus. Wattage demand will already be established on a pre-built system. StarTech can bail you out here once again.

Butter needed one PSU ready to go, a quick search and I snagged up a used model for $20. Again, you really need to know what you're looking for: Butter's PSUs will only fit another ProLiant G6, so best to triple check before clicking "buy."

As a pre-built server, Butter already comes with nearly all the bells and whistles we could want built-in. That said, Butter is expecting 3.5" drives and its front bay loader is designed for this.

While we *could* be cavemen and just stick the SSDs in there on faith alone, we're instead going to invest in a couple 3.5" drive adapters to keep things professional.


Controllers, Firmware & BIOS (O My)

Ilopwr1.JPG
Ilopwr2.JPG

BIOS options for a pre-built server can be extensive, but good news: the defaults are almost always already tuned for optimal use. Thus, your visitation to the BIOS may be limited to selecting boot the order, enabling virtualization, or supplying owner info & asset info, 'cause why not have your server say "butts" on startup?

I always spent some time with the server's onboard RAID controller for the front drive bays. For this initial pass at Butter (heh), all that was needed was a single disk array. About three keystrokes and we're done: RAID 1 (at least according to the controller; it's really JBOD), single disk.

Another big cause for attention on a true-blue server would be any built-in controllers or diagnostics items, in Butter's case this would be the iLO2 controller. This sucker gives us access to a lot of low-level options for the server, accessible over TCP/IP with a username/password.


Boot It Up!

Ubuntu: Linux for the rest of us.

As for installing the OS (as mentioned, chances of finding an intact server OS on a thrifted system is null), this one's easy, 'cause everyone's already done it! Linux is simple to install, and with Ubuntu as an example, is well covered here.

Your options for configuring your server are endless and best guided by your needs. Choose the services, roles & features that are going to best help your organization and plan how you wish to deliver these: virtualization, for example, is an awesome way to get the most from your hardware without turning it into a jumbled mess. If this sounds good to you, consider configuring a hypervisor for your server! Microsoft's Hyper-V Server, Citrix's XenServer or the XCP-ng alternative are all great hypervisor choices.

Specific to Butter, we've decided to first give it a spin with Ubuntu Server as a "health check." Usually I'll do this just off of a USB drive, but as an extra step to ensure the front bay array works, I've decided to install Ubuntu Server directly to the SSD using standard installation media.

Here's a quick tip for anyone new to Linux & looking to use to aid in their thrifting: a common Linux use-case is the "Live CD." The name is a misnomer these days - a Live CD distribution is not limited to compact disc, and the term "CD" really just means to refer the "image" format that's used to burn compiled data onto various mediums like USB drives, DVDs or... well, CDs.

The "CD" in the name also tends to mean the distribution is light weight and able to fit entirely on a single 700MB compact disc. This is how I am able to boot to Linux from a USB drive; if you're in a store that will grant you access to test the machines before you buy, a handy USB drive containing a Linux Live CD is the only tool you need.

As a side note, Microsoft also has its own variance of the Live CD known as the "Preboot Environment" or "PE" for short. This is basically a Windows Command Prompt that runs off a RAMDisk, generated when you boot from the PE media - it's used by Microsoft itself as part of the Windows installation framework. With the right software, you can create your own Windows PE and run some software right from that Command Prompt.

Appliances

Hardware

Gigabyte GA-C847N-D.jpg, an example of a small board with a low-power CPU built in & dual NICs.

Motherboard & CPU

The Intel Celeron: used to be a joke, now it's a handy micro-use chip.

Every couple of years, a new built-in CPU combo board series will be released with several manufacturers offering their own configurations and bringing much joy to the DIY-appliance crowd. These boards are often under $100 and have plenty of power to run things like a Windows server or any Linux hypervisor offering. They can even stand in as a small desktop, but your mileage may vary.

A common series of traits found amoung these built-in CPU combo boards would be:

  • Micro-ITX or similar tiny form-factor
  • Dual DIMM slots (and no more..)
  • Dual NIC ports
  • Intel Atom, Celeron or similar low-power draw CPU
  • A hard cap on expansion, but it is available
  • Moderate performance, but flexible

Memory & Storage

G.Skill memory is inexpensive, reliable & has a great warranty.
SanDisk makes a decent mid-range SSD for less, great for appliances.
Inland makes a great, inexpensive SSD.

First thing's first: when it comes to a server, not all memory is equal (or even an option). Most "real" servers are going to need ECC and/or Registered memory. Again, checking the manual for your selected system to see what it prefers is going to pay off in this sense.

Without getting too technical, ECC stands for Error-Correcting Code. This means the memory is able to make adjustments and correct common cases of data corruption on the fly. This makes the memory more reliable and, in a sense, faster. Registered memory has a buffer cache built-into each DIMM; again, the idea here is reliability and speed.

When it comes to vendors, the best depends on who & what the server is for. Don't care if it there's downtime? Buy budget memory from a warehouse vendor, maybe it'll keep kickin', maybe it'll burn out. Else, invest in memory with a good warranty; the "good" stuff will still be inexpensive as long as it's not flashy.

If you're on a budget, many retailers will also sell refurbish/rebranded memory for older systems or servers in "bulk" (16GB kits or more). The price point for such a kit that will fit Butter is good, but our steepest cost yet: just under $200 and we have 40GB of DDR3 memory to feed our two Intel Xeons. The brands are mixed, but all the sticks check out and play nice.

Primary storage for a server should be SSD based these days; fast read and write times are expected of any system intended to serve the masses, so an SSD will be optimal. We can also take advantage of mechanical HDDs for storage of data that doesn't need to be quite as zippy. That said, a lot of software storage will help the experience by using a portion of your SSD for automatic caching, so in a way we get the best of both worlds.

Vendors for storage don't vary as much these days, frankly most HDDs all come out of the same, few factories. That said, go with the ones you know that have good warranties: Western Digital, for example, is one I recommend only for their warranty support. Drives die - unless you wanna throw money away, anticipate this "feature" and get a company who'll bail you out when your array is about to crumble.

If you are on a budget, there another alternative. "White label" disks are available from places like Amazon at very reasonable prices. These drives are built by major manufacturers, but stripped of their branding and made generic for use in pre-built consumer PCs. 3TB, 7200RPM for $40; hard to argue with.

Chassis, PSU & Other Diddies

The "Pedestal" or Tower format, less common for servers.
Server PSUs are often proprietary & oddly shaped.

Your chassis should fit the size and intent of your project. For a server, we want it to conform either to the intent or the environment, meaning we must design it for a rack or the hardware within. For a true server, we're likely to encounter everything already in the appropriate chassis, so look for the style that suits your project best: a two-post, rack-mount chassis is not a bad way to go for smaller, lighter systems. For heftier hosts, a "tower" or "pedestal" design is preferred that's got enough room for proper cooling.

A reliable vendor for projects like this would be StarTech. Make sure candidate chassis have enough ports and room for all the bits you wish to add. Deckard's chassis will be a 2U front-facing with handy grips for easy maintenance (though I don't intend on opening this sucker much). The import thing for this project is to get a case that has room for a lot of hard drives. A lot of 2U cases will have a decent amount of storage space in the front, thus this is a good option for a decent amount of space in a compact format.

If your server chassis already has a PSU, great! You saved yourself a headache. Else, if you're selecting your own PSU, triple check the form factor and see if you can find high-res pictures of the specific model if it's an odd size or shape - sometimes things just don't line up with the smaller cases. When it comes to pre-built servers, PSUs are often proprietary and purpose-built, be sure they're available at a price affordable to you if you plan on investing in such a system.

Cheaper PSUs these days are a lot more reliable than they were 10 years ago, but efficiency will still cost you. Even so, I suggest going with the most efficient - look for 80 Plus. Wattage demand will already be established on a pre-built system. StarTech can bail you out here once again.

A 2U chassis allows for a full, desktop-sized PSU, which is great because these are the most common and are often inexpensive. 300w is more than enough to run a bunch of disks and a little motherboard.


Controllers, Firmware & BIOS (O My)

Most built-in CPU combo boards will have a basic desktop grade BIOS or UEFI experience. You'll get to tweak some minor things but don't expect ultimate control or anything like out-of-band management.

The default settings are largely automatic and what we want anyhow.

The RAID controller will be used mostly for pass-through use, so we don't really need to access its config - by default it will pass the disks on to the OS. This is a handy setup if you intend on using software defined storage, which frankly is a lot easier and safer than hardware RAID.