Dee-Nee Forums

General => News => Topic started by: Gantry on 06/02/06, 10:33:04 AM

Title: Site back up
Post by: Gantry on 06/02/06, 10:33:04 AM
Sorry about that, not sure what went on.  Looked over to dee-nee and the out-of-memory killer was running, I couldn't login or reboot.  Waited to see if the OOM killer would work and I ran out of time so I hard reset.  Then the fsck had to run...

I really need to get a journalling filesystem installed on dee-nee...

Title: Re: Site back up
Post by: Nacho on 06/02/06, 10:35:44 AM
Gantry broke the internets.
Title: Re: Site back up
Post by: Guy Smiley on 06/02/06, 10:46:59 AM
hey, the site was down a few minutes ago.  whats wrong with the servers?
Title: Re: Site back up
Post by: Gerlost on 06/02/06, 10:48:08 AM
It's running kind of slow now...
Title: Re: Site back up
Post by: TβG on 06/02/06, 10:48:24 AM
Quote from: Gantry on 06/02/06, 10:33:04 AM


I really need to get a journalling filesystem installed on dee-nee...



i heard a program called ultimate7 is pretty good
Title: Re: Site back up
Post by: ultimate7 on 06/02/06, 11:30:50 AM
Yeah but when it's down I can't get through to post, I should have email Gantry (does his email still go through to the Dee-Nee address when the site is down?)
Title: Re: Site back up
Post by: Gantry on 06/02/06, 03:20:48 PM
Email is housed on a different server, email will get through...
Title: Re: Site back up
Post by: Big Hath on 06/02/06, 03:24:14 PM
Gantry has the technology, he can rebuild.
Title: Re: Site back up
Post by: Gantry on 06/02/06, 05:43:16 PM
I rebooted the server to get the memory back to 384M, hopefully nobody noticed...
Title: Re: Site back up
Post by: nightwulf on 06/02/06, 07:10:24 PM
Quote from: Gantry on 06/02/06, 10:33:04 AM
I really need to get a journalling filesystem installed on dee-nee...

You know you can convert ext2 to ext3 in place, right? Takes like |----| that long.
Title: Re: Site back up
Post by: BDawk on 06/02/06, 08:03:10 PM
Quote from: nightwulf on 06/02/06, 07:10:24 PM
Quote from: Gantry on 06/02/06, 10:33:04 AM
I really need to get a journalling filesystem installed on dee-nee...

You know you can convert ext2 to ext3 in place, right? Takes like |----| that long.


With all those cell phones on his belt ringing all the time, how can you expect him to rmember that?
Title: Re: Site back up
Post by: JerryD on 06/02/06, 08:48:03 PM
Quote from: Gantry on 06/02/06, 05:43:16 PM
I rebooted the server to get the memory back to 384M, hopefully nobody noticed...

I could have sworn it was 380M.
Title: Re: Site back up
Post by: fknmclane on 06/02/06, 08:57:13 PM
Whatever it takes.
Title: Re: Site back up
Post by: Gantry on 06/02/06, 09:27:29 PM
Quote from: nightwulf on 06/02/06, 07:10:24 PM
Quote from: Gantry on 06/02/06, 10:33:04 AM
I really need to get a journalling filesystem installed on dee-nee...

You know you can convert ext2 to ext3 in place, right? Takes like |----| that long.

Assuming your kernel has support for it.


natas% more /proc/filesystems                              gantry at 21:24 in ~
nodev   rootfs
nodev   bdev
nodev   proc
nodev   sockfs
nodev   tmpfs
nodev   shm
nodev   pipefs
        ext2
nodev   ramfs
        iso9660
nodev   devpts
nodev   usbdevfs
nodev   usbfs


Edit - Looks like I have /sbin/mkfs.ext3, perhaps I don't understand /proc/filesystems.  Will be figured out when I'm not drinking beer (or when nightwulf does it for me)
Title: Re: Site back up
Post by: nightwulf on 06/02/06, 09:54:15 PM
I'd happily offer to recompile the kernel, but you'd need to be at the server with a backup on disk in case something fucks up. That and you probably don't have kernel sources lying around with your girly-ass RedHat distro!
Title: Re: Site back up
Post by: Big Hath on 06/03/06, 12:05:58 AM
last time I recompiled a kernel, I woke up in a dusty alley in Tijuana
Title: Re: Site back up
Post by: Gantry on 06/04/06, 10:45:31 AM
Quote from: nightwulf on 06/02/06, 09:54:15 PM
I'd happily offer to recompile the kernel, but you'd need to be at the server with a backup on disk in case something fucks up. That and you probably don't have kernel sources lying around with your girly-ass RedHat distro!

Well, grabbing the kernel source isn't a problem nor is apt-get to get redhat's kernel source.  I can compile the kernel, I'm just lazy.  Two things I would definitely like to add are ext3 & support for USB hard drives (and whatever USB 2.0 controllers I decide to put in there) - it would be nice to have a better backup system in place. 

Perhaps we should have another "taco tuesday maintenance night" like last time where we got the forums/wikis upgraded in one night.  I can work on another wiki upgrade (including mysql 4/5 install for the latest mediawiki upgrade) while you can get kernel 2.6 up and running.  I'll do it onsite incase the kernel boot goes all sad on us.  What say you? 

Fuck, I may put another HD in there as well to do software RAID.  One more point of failure in case the dee-nee hard disk blows up...

Title: Re: Site back up
Post by: Gantry on 06/04/06, 10:47:02 AM
And before I get a scolding from nightwulf, I should clarify that I would install a separate mysql 4/5 instance on a different port and leave the forums on the old mysql.  Assuming that's possible of course, I haven't looked into it that much...
Title: Re: Site back up
Post by: nightwulf on 06/04/06, 08:44:01 PM
Why do that? SMF runs fine on MySQL 4/5 as far as i know.
Title: Re: Site back up
Post by: Gantry on 06/05/06, 12:01:27 PM
It does, but I wasn't intending on doing an in-place upgrade.  Maybe get it running and move one db at a time over...

So how does your schedule look?  If you are free tomorrow then maybe I'll plan a "taco night" (assuming we have a taco) at the office to start upgrading....
Title: Re: Site back up
Post by: BDawk on 06/05/06, 12:06:01 PM
Hey! Talk nerd stuff on your own time!
Title: Re: Site back up
Post by: Mr. Blubbo on 06/06/06, 03:19:02 PM
After you reconfigure the kernels, can I eat the unpopped ones?
Title: Re: Site back up
Post by: Gantry on 06/08/06, 09:40:40 AM
Site went down again yesterday pm, thanks to Ultimate for paging me, as I was at a customer.  I definitely have to figure out a solution to this problem.  We pretty much run out of available memory (RAM) on the system if it gets too busy.  The problem is that this shitbag motherboard often only reports 256MB of the 384MB in the system.  It takes many reboots to get it fixed and if I'm booting remotely (like yesterday, was at a customer) I simply cannot wait...

So Dee-Nee needs a hardware upgrade.  Not sure if I want to do a whole shebang upgrade at this point or simply find the cheapest and easiest way to get more RAM in here without changing the software at all...

I don't know why the motherboard (http://www.crucial.com/store/listparts.asp?Mfr%2BProductline=ASUS%2B+Motherboards&mfr=ASUS&tabid=AM&model=P2-99&submit=Go) is so picky with RAM, it should take basic PC-133 memory.  It's a picky little bitch though, and if I replace the mobo I likely have to replace the processor since Slot1 hardware ain't terribly easy to find.  Which means I'm pretty much swapping out every piece of hardware in it.  If I do that I might as well go new and start an OS install from scratch. 

So that is where dee-nee stands for now.  To make sure I don't exhaust memory, I'll stop using it as a workstation from here on out.  Going into X definitely saps the memory and if I forget to close out before I leave, it's much more likely to OOM...

Title: Re: Site back up
Post by: Gantry on 06/08/06, 10:23:01 AM
I found that magical combo of RAM sticks that gives me the full 512MB that this system allows.  This is huge...
Title: Re: Site back up
Post by: Guy Smiley on 06/08/06, 10:32:07 AM
current server details please.  Slot1 is that intel?  I can never remember.  My "server" is an athlon thunderbird 900mhz Slot (1 or A, whichever is AMD), and I'm able to put 768mb or RAM without any problems...
Title: Re: Site back up
Post by: nightwulf on 06/08/06, 07:50:30 PM
What exactly are the specs of the server? Hell, I probably have something comparable lying around ...
Title: Re: Site back up
Post by: Gantry on 06/08/06, 09:05:28 PM
Dee-Nee Server Hardware Specs

* Pentium III - 600mhz (Slot 1)
* 512MB (as of today, usually 256 or 384) of PC100 SDRAM
* ASUS P2-99B motherboard
* 20GB Maxtor IDE hard drive (I think 7200rpm)
* Generic CD-RW for nightly backups where I hardly swap discs
* $5 realtek 10/100 NIC
* ATI Rage IIc AGP video

Dee-Nee Software Specs

* Redhat 7.3 with full updates (including fedoralegacy)
* 2.4.20-43.7 kernel
* Apache 1.3.27-9
* MySQL 3.23.58-1.73.9
* Php 4.4.2 (hand compiled)
* Perl-5.6.1-38.0.7.3.3
* SMF 1.1 RC2
Title: Re: Site back up
Post by: Gantry on 06/08/06, 09:11:14 PM
Site's been up since 10am and we have 63MB of free RAM and zero swap used.  Massive, massive bonertown...
Title: Re: Site back up
Post by: nightwulf on 06/08/06, 09:16:47 PM
Hell, other than the RAM, the PC you handed me in Columbutt has better specs. I could throw that one back together, and find some RAM for it ...
Title: Re: Site back up
Post by: Gantry on 06/08/06, 09:32:06 PM
It's not worth swapping our for a marginal gain in speed...  Heck if the PCI ordering is flaky or the IDE is enumated differently I may have to troubleshoot and stuff.  Plus I'd have to reconfigure my sound and stuff...  I have a few machines that are right around that range, heck sometimes we throw them out.  I fear change, that's all...

If I do anything that involves a mobo swap, I'm going with a fresh install of CentOS 4.  But the RAM upgrade makes me so happy that Dee-Nee will last in its current form for at least another 7-8 years.  I would love to do software RAID though, a hard disk crash would be bad...
Title: Re: Site back up
Post by: fknmclane on 06/08/06, 11:47:39 PM
Quote from: Gantry on 06/08/06, 09:32:06 PM
Dee-Nee will last in its current form for at least another 7-8 years.

You and I will have about a million million posts by then.
Title: Re: Site back up
Post by: malnuboy on 06/09/06, 12:06:34 AM
I'll JPeg that over to you
Title: Re: Site back up
Post by: fknmclane on 06/09/06, 12:13:16 AM
Quote from: Gantry on 06/08/06, 09:40:40 AM
Site went down again yesterday pm, thanks to Ultimate for paging me, as I was at a customer.

I was actually online for this one.  My whole day at work was ruined because dee nee was down for a whopping half hour.  Pretty nuts.
Title: Re: Site back up
Post by: nomaaa on 06/09/06, 02:11:51 AM
Quote from: fknmclane on 06/09/06, 12:13:16 AM
Quote from: Gantry on 06/08/06, 09:40:40 AM
Site went down again yesterday pm, thanks to Ultimate for paging me, as I was at a customer.

I was actually online for this one.  My whole day at work was ruined because dee nee was down for a whopping half hour.  Pretty nuts.

i came home to this. and boy was i pissed. it almost ruined my whole day.
Title: Re: Site back up
Post by: Gantry on 06/09/06, 08:18:32 AM
Swap finally got touched, but the system is moving along quite nicely with the 512MB in there.  The golden age of Dee-Nee is upon us...
Title: Re: Site back up
Post by: Mike D. on 06/13/06, 01:28:01 PM
PAX DEE-NEE