Greg Hewgill ([info]ghewgill) wrote,
@ 2005-05-04 13:02:00
Previous Entry  Add to memories!  Tell a Friend  Next Entry
Entry tags:computers, web

per-process bandwidth accounting
Dear Lazyweb,

I want to track bandwidth used by processes on my colo server on a per-process basis. I'm running Apache, BitTorrent, Freenet, and miscellaneous other things like automated backups. I have a monthly bandwidth cap of 1800 GB. I'm running FreeBSD 5.3. What are my options?


(20 comments) - (Post a new comment)


[info]lucky_mcgee
2005-05-04 06:20 pm UTC (link)
Have a donut and coffee, and pay somebody else to do it? =oD

heh..sorry, that's all I've got. =oþ

(Reply to this) (Thread)

[info]decibel45
2005-05-04 08:08 pm UTC (link)
I'll add greg to the list of people who will be helping to toss your butt out of a plane this summer. (Something he's already done.)

(Reply to this) (Parent)(Thread)

[info]lucky_mcgee
2005-05-05 08:19 am UTC (link)
OH NO!! There are TWO of you?!?! =oO

I'm SURE to be killed.

I didn't realize you guys were "real life" type friends. You guys should post more so I can figure all this stuff out.

At the rate either of you post, and as much content as you put into your posts, I won't know your favorite food, drink or color until...oooh....maybe 2007...? ;oP

(Reply to this) (Parent)(Thread)

[info]ghewgill
2005-05-05 01:43 pm UTC (link)
Well I first encountered your journal when I was over at [info]snaxxx's place with [info]decibel45 and [info]_fool, and we were planning SXSW 2005 Nuclear Taco Night. I can't find it now, but you invited people to call your voice mail and leave a message. We did, collectively, and I recall [info]snaxxx mentioning that it made an impression on you. :)

You definitely should plan a trip to Austin sometime so we can throw you out of an airplane, feed you [info]nucleartacos, have some homebrew (and make some more), and so on. And a new thing - last night we made fresh mozzarella cheese (first time for all of us) that was absolutely to die for.

(Reply to this) (Parent)(Thread)

[info]lucky_mcgee
2005-05-05 06:50 pm UTC (link)
Ooooh YOU were the Cartman??? xoD hahaha! That was great. My favorite message of the night. And it's still on my machine. I refuse to erase it.

I'll be making my annual summer road trip in a few months, and am actually in the process of planning out my locations I'll be going to right now, and Austin is already on the list!

But I can't make any promises about jumping out of planes. heh. However, I'm more than game for tacos drinking and all the cheeziness you people have to offer! =oD

(Reply to this) (Parent)(Thread)

[info]ghewgill
2005-05-05 10:02 pm UTC (link)
[info]decibel45 is the loud one, he was cartman.

I don't think we've tried drinking the tacos before, though. You'll have to let us know how that goes.

(Reply to this) (Parent)(Thread)
'scuse me for forgetting a comma.
[info]lucky_mcgee
2005-05-05 10:05 pm UTC (link)
tacos, drinking...

Better!? =oP

(Reply to this) (Parent)

[info]decibel45
2005-05-05 09:57 pm UTC (link)
You know [info]snaxxx has jumped out of a plane as well, right? :>

(Reply to this) (Parent)(Thread)

[info]lucky_mcgee
2005-05-05 09:59 pm UTC (link)
OH GOD NO. The pressure.

Can't we just eat nuclear tacos instead?!? =oD

(Reply to this) (Parent)(Thread)

[info]ghewgill
2005-05-05 10:09 pm UTC (link)
Don't worry, the pressure isn't a problem. At roughly 1 inch of mercury per thousand feet, the pressure at exit altitude is just over half what it is on the ground. :)

(Reply to this) (Parent)(Thread)

[info]lucky_mcgee
2005-05-05 10:12 pm UTC (link)
You're not helping the fears here.

(Reply to this) (Parent)

[info]decibel45
2005-05-05 10:17 pm UTC (link)
Sure, after we get back on the ground. :)

(Reply to this) (Parent)(Thread)

[info]lucky_mcgee
2005-05-05 10:28 pm UTC (link)
I can't believe for like 0.009 seconds, after I read that, I actually entertained the idea.

Thank the good lord for reality. heh.

(Reply to this) (Parent)

[info]ghewgill
2005-05-05 07:16 am UTC (link)
As you will no doubt see as this project proceeds, that is so not my style. :)

(Reply to this) (Parent)(Thread)
So I'm not one of the cool kids...
[info]lucky_mcgee
2005-05-05 08:17 am UTC (link)
heh...I'm sure it isn't, but like I said, it's the best I've got. :oþ

(Reply to this) (Parent)

[info]paradox0220
2005-05-04 06:51 pm UTC (link)
That is a cool idea. Track and limit bandwidth per process (or process type, all httpd clumped together in stats and control). Time for greggie to do some kernel hacking? :)

(Reply to this) (Thread)

[info]ghewgill
2005-05-05 07:17 am UTC (link)
I don't think it's going to need kernel hacking. I'm pretty sure I can do this in all user mode code.

(Reply to this) (Parent)

[info]decibel45
2005-05-04 08:08 pm UTC (link)
The only thought that comes to mind (and this is a bit ugly, so hold on to your hat) is this: NAT let you forward ports from the outside interface to a specific IP behind NAT. So, if you can collect SNMP data on aliased IP addresses, you can run NAT and have it point at a bunch of aliased IP's.

Another possibility is afaik ipfw will let you filter by user, so you might be able to do something with that.

Maybe your best bet would be to write some code that ties into tcpdump and tracks how much data went where.

(Reply to this) (Thread)

[info]ghewgill
2005-05-05 07:20 am UTC (link)
Well see I've got this great colo server that's not behind nat or anything, it's plugged right into the internet. It would be a real shame to set up some kind of virtual nat layer on that machine!

I do run the major services on my machine under separate accounts, so perhaps I could look at the per-user accounting angle for this. Maybe a jail with the same root directory.

Using data from bpf (berkeley packet filter, which tcpdump uses on freebsd) wouldn't tell me where the data comes from, except by port number which is annoying for dynamic services like bittorrent and freenet.

(Reply to this) (Parent)(Thread)

[info]decibel45
2005-05-05 02:39 pm UTC (link)
I'm pretty sure jail will want you to setup seperate IPs for the jails.

If you use the per-user/group filtering in ipfw, you could then use tee to send it to a divert port, which could be a daemon that does nothing but count bytes (and packets). It might not be 100% accurate compared to SNMP because it would ignore lower layer overhead.

(Reply to this) (Parent)

(20 comments) - (Post a new comment)

Create an Account
Forgot your login or password?
Login w/ OpenID
English • Español • Deutsch • Русский…