The actual site
The actual site is the tasvideos.org website and its subdirectories, including the forums.
It contains the following properties:
― A webroot wherein the site engine is executed from. In the same webroot, is the forums, and all the backend functions (such as the SOAP API) of the website.
― Two MySQL databases: one for the actual site (nesvideos_site) and one for the forums (nesvideos_forum).
― A memcache cache process.
I'd be interested in taking the work with simply administrating the site.
Maintenance requirements:
― Your Internet connection should allow incoming connections to port 80 (http). (And likely 22 (ssh), for the developers.)
This can be arranged, although it is not true currently.
― Absolutely trustworthy and keeps confidential information confidential.
I'd like to think I'm trustworthy, but that's your call.
― Ensuring that all software is running and operational.
I've worked as a Linux admin before. And if you'd like a resume I can e-mail one to you.
― Ensuring that the Internet connection (network) is capable enough and robust.
I have a capable enough internet connection currently, and I will have to upgrade to allow port 80 opened.
― Ensuring that the hardware is sufficient for the site. The current site has a Intel Core2Quad (64-bit 4-core 2400 Mhz) and 6 GB of 800 MHz RAM. It is well enough to run everything mentioned in this post.
My current workstation computer (oberus) is a dual core AMD 64-bit with 6GB of 800 MHz RAM. I can make it the server and turn one of my other computers into my workstation.
― Ensuring that there is enough disk space. The website (including forums) requires about 8 MB of disk space; the TASSnapShot requires about 30–50 MB, and the MySQL database requires 1 or 2 gigabytes.
I'll need to get a RAID card if I'm going to turn oberus into a server. But I should have the disk space covered.
― At least the following software are required on the site: Apache version 2.2 or later, PHP version 5.2 or later, MySQL version 5.0 or later; the following PHP extensions: Curl, Eaccelerator, Gd, Memcache, MySQL, Tidy, XSL, SOAP.
This can happen.
― Give maintainers of the site source code the access to update the site, and to access the databases directly.
This can also happen. I can also arrange svn or whatever per Nach or whomever until the wiki code engine is done.
― Your bandwidth should be such that your server can upload something like 20―50 kB/s. More is better, of course, and especially with the large Movies-*.html pages (which vary from 200 to 600 kilobytes), users will appreciate it if the pages load quickly.
My current upstream is ~150kbps, this may change with the required internet connection upgrade.
Build a man a fire, warm him for a day,
Set a man on fire, warm him for the rest of his life.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Bisqwit wrote:
Nach's idea is splendid, but it doesn't solve the problem of accidentally using pathnames outside the allowed webroot and rendering them to the browser, and the such.
You can prevent that with proper filtering.
IIRC, Apache allows locking files and symlinks read to a particular root.
However for more security, you can run Apache inside a chroot. Since you have multiple sites, you'll need a bit more trickery. Run more than once instance of Apache, have the TASVideos site run on the chroot'd Apache which runs on a different port. Have tasvideos.org:80 on the main Apache do a "Location: http://'.$_SERVER['HTTP_HOST'].$other_port.$_SERVER['PHP_SELF'].$getVariables".
If you want them both on port 80 with a chroot. You can setup a tunnel for the site on port 80 to foward the request onto the other port and pass on the results, leaving the other Apache locked well.
If you don't want a chroot, you'll need to setup a PHP prefilter to catch everything beyond Apache's level. All file system access by PHP will need to be passed through realpath(), and then see if your list of allowed roots matches the beginning until a slash of the realpath.
Or perhaps you can try that mod to run PHP scripts as a different user. In which case you have that other use have very limited permissions, and rely on UNIX and your partitions setup properly.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I'll write details about the source code, database and testing procedures to those volunteering for development later ― as soon as I get the frameworks for updating that stuff finished.
I'll continue hosting the main site and images until a really promising hosting situation emerges ― which I still hope happens within the next half a year. I switched yesterday to using suexec and I have cleared up the permissions a bit. Possibly I'll also experiment with using a LD_PRELOAD that restricts access, too.
The tracker could be moved at any time, though. It's the least complicated piece :)
I was wondering Bisqwit, will you be picking anyone or are you just going to accept applicants?
Since you are the founder (and thus the longest member here), I would assume you have a few candidates of your own you would probably feel comfortable handing aspects of the site over to? Just curious of course. :)
Thanks!
I had no idea it was so involved to maintain tasvideos. It is very gracious of you to in advance make sure it carries on undisrupted. But even more gracious to run it all along. Thanks Bisqwit.
I think I can host the image server, that one looks like it doesn't require shell access to maintain and uses less than 300 gigabyte of bandwidth per month?
Bisqwit, I just wanted to wish you the best of luck in whatever you do in the future. I truly think you are this community's greatest asset.
This really belongs in the "new site layout" thread, but I'll just ask it here since it was briefly mentioned in the first post. For future reference, where is the donate page now? I can't find it anymore, but that's probably just my fault.
Bisqwit, I just wanted to wish you the best of luck in whatever you do in the future. I truly think you are this community's greatest asset.
This really belongs in the "new site layout" thread, but I'll just ask it here since it was briefly mentioned in the first post. For future reference, where is the donate page now? I can't find it anymore, but that's probably just my fault.
Thanks for the feedback.
the donation page url still works, but I removed links to it since I don't consider it fair to keep them if I'm about to have it hosted by someone else. if you want to use it, I can PM you the link or Google-search might still find it.
If I can get the security issues sorted out, I can continue hosting the main site + forums. (But the tracker and the image server could still easily be moved. (Well, the image server only after the API is created to it.)) It's really only the development & admin tasks that I'm concerned of with regards to the amount of my time that it needs.
the donation page url still works, but I removed links to it since I don't consider it fair to keep them if I'm about to have it hosted by someone else.
It seems like you should put the donation links back up, but have it go to a different account which you can transfer to the new host/owner when he/she takes over.
How fleeting are all human passions compared with the massive continuity of ducks.
I ain't ready for any of the big stuff, but...
I can help out with getting the avi files onto archive.org.
I just need some help getting the files to my computer.
I can help out with getting the avi files onto archive.org.
I just need some help getting the files to my computer.
lol, welcome to my world. I uploaded as many AVIs as I could, but I stopped when I realised that I wasn't able to get the files in the first place.
I actually managed to download a couple more of them in the last month, I might as well upload them, so thanks for the heads up, I guess.
edit:
Since it has been some time since the last time I mirrored something on archive.org, I don't know who I should send a PM to, I used to contact either mmbossman or Raiscan, but I don't know who of them is more active nowadays, so I'll just drop the links here, and whoever wants to edit them into the page, be my guest.
This goes hereThis goes hereThis goes here
3 more to come, but later.
I edited the first post by clarifying some things about the schedule and the required responsibility shifts.
Re: Development -- developer access will be opened once I get the site-update system developed and finished.
The API for the image/media server has now been written. The main site accesses the image/media server exclusively through the API, meaning that the image/media server can now be migrated at relative ease when someone offers to host it. Who will?
It currently weighs 463 MB, with the bulk of that room being taken by site snapshots and various short demonstration AVIs I have occasionally linked to on these forums.
Just posting a bump to remind people that this is still on-going.
In November, I wrote that "The desired timeline for this all is that by the summer of 2009, everything has been moved."
It's getting to the summer now, and all the stuff is still on my server. Why is it so difficult to get someone to host a PHP site?
As the summer deadline is crossed, things are now shifting from a "wish for transition" to a "burden of liability" phase. From tomorrow on, re: topic of hosting, I may do a sudden action if I see a choice even if it's not a perfect one. (Still probably in concordance with http://tasvideos.org/forum/p/142644#142644.)
Just to recap it all, hosting is needed for:
1. Image server (footprint 500 MB, though it can be reduced into 40 MB if we scrap demonstration AVIs, tassnapshots, autoscreenshots and my WIPs; needs a webserver with PHP with SOAP extension, no databases
2. Bittorrent tracker (a webroot containing .torrent files downloadable, and with PHP with SOAP extension; BitTorrent tracker software running (we use bnbt); and firewall configured such that incoming connections are allowed to ports 80, 6967 and 6969; no databases)
3. Main site (two MySQL databases. LZMA compressed SQL backups thereof total about 60 MB, but the active database with all indexes and stuff needs about 1 or 2 GB; a website with a number of PHP extensions installed; memcache installed; give the maintainers write access to the webroot, which needs to have less than 20 MB of space).
The problem is it isn't a PHP site.
A PHP site is a site that can be archived and moved somewhere else without any trouble.
Just based on the wiki code I recieved from you, it requires a very specific setup involving having certain cache utilities installed and other things. Just getting the code to work on my server required gutting a large amount of it, and even then it was still very subpar in terms of stability and working (some things would crash it).
If it was as simple as copying a MySQL database, a PHP site, and editing some .htaccess files, then no problem. But as it is, it's not something most web hosts would be able to handle with standard packages, and few people are privledged to have an ISP that not only allows connections on port 80, but also allows actual server hosting and static IPs (most TOSes I've read are very adamant about not hosting medium/high traffic web servers).
If I had the capability to host it on my DreamHost account, I would without question. But between lacking SOAP, missing memcache, and several other things, it isn't possible.
I moved the main site out from my own server, to the company-where-I-work-at's server.
Not a perfect solution ― I think the response time became slightly worse here for some reason. But at least it's away from my own server now. Only, if the company decides to remove it... tough luck. (I do have backups, but.) And it does not resolve the development access problem, rather on the contrary.
Still waiting for someone to take it.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
I now have my own server to host stuff on. The question is, does it have the capacity for this site?
I can provide 2GB HD space. 30 GB of transfer a month. RAM and CPU usage by scripts has to remain pretty low. Would that be enough to cover things?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I now have my own server to host stuff on. The question is, does it have the capacity for this site?
I can provide 2GB HD space. 30 GB of transfer a month. RAM and CPU usage by scripts has to remain pretty low. Would that be enough to cover things?
The CPU usage is very low (at least if you have RAM for disk-caching most of the database), RAM usage however can peak a bit (as in, the default memory limit of 16 MB in PHP is not enough), and add the space used by memcache to that.
The disk space should not be a problem.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Bisqwit wrote:
Nach wrote:
I now have my own server to host stuff on. The question is, does it have the capacity for this site?
I can provide 2GB HD space. 30 GB of transfer a month. RAM and CPU usage by scripts has to remain pretty low. Would that be enough to cover things?
The CPU usage is very low (at least if you have RAM for disk-caching most of the database)
Well, maybe I'm wrong on this, but I imagine that most of the main site can be stored in mod_disk_cache, and only updated twice a day or something. Which is good for CPU usage.
Bisqwit wrote:
RAM usage however can peak a bit (as in, the default memory limit of 16 MB in PHP is not enough), and add the space used by memcache to that.
The disk space should not be a problem.
I don't have a lot of RAM on my server. I have to check but I think I limited PHP itself to 32MB or 64MB, and then gave another 32MB to XCache.
How do you think that would work?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Of the three components that can be each set up on a different server (or the same):
BEGIN COMPONENT: TASVIDEOS MAIN SITE
Tasvideos main site :: Webroot: less than 5 MB [Mandatory]
Tasvideos main site :: MySQL databases: at least 500 MB (difficult to say due to InnoDB storage) ― any way less than 2 GB [Mandatory]
END COMPONENT
BEGIN COMPONENT : TASVIDEOS MEDIA SITE
Tasvideos media site :: Editor-selected screenshots :: about 12 MB [Mandatory]
Tasvideos media site :: PHP code for integration with main site :: about 1 MB [mandatory]
Tasvideos media site :: Automated screenshots :: about 35 MB [Optional]
Tasvideos media site :: Article images stored here :: less than 2 MB [Mandatory]
Tasvideos media site :: Most recent TASSnapShot :: about 50 MB [Recommended]
Tasvideos media site :: Older TASSnapShots :: less than 100 MB [Optional]
Tasvideos media site :: DOSBox rerecording zips :: less than 5 MB [Recommended]
Tasvideos media site :: WIPs and demo AVIs stored here (stuff linked to from a couple of Bisqwit's posts on the forums, accessed rarely) :: about 350 MB [Optional]
END COMPONENT
BEGIN COMPONENT: TASVIDEOS TRACKER
Tasvideos tracker :: active torrents :: about 7 MB [Mandatory]
Tasvideos tracker :: obsolete torrents :: about 7 MB [Recommended]
Tasvideos tracker :: tracker software :: about 10 MB [Mandatory]
Tasvideos tracker :: PHP code for integration with main site :: about 1 MB [mandatory]
END COMPONENT
BEGIN COMPONENT : IRC BOTs (not available for migration)
NesVideoAgent :: ircII scripts :: less than 1 MB
BisqBot :: ircII scripts :: less than 1 MB
BisqBot :: auxiliary programs and their file-based data :: about 180 MB
BisqBot :: databases accessed :: a lot and some more
END COMPONENT
I don't know how much it could cost, but wouldn't it be possible to host this on a virtual private server ?
How much would it cost, and could this be driven by the community. I know it's asking for donation, but maybe it would be possible to generate enought donation to hold the website alive without the problem of splitting the server across three diferent server.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Bisqwit wrote:
Nach wrote:
How do you think that would work?
Not sure. The database (MySQL) needs considerable amounts of RAM to be efficient.
Well, my server has 128 MB free for Apache and MySQL. Although I'm hosting other sites on it too.
Bisqwit wrote:
I could buy you RAM too...
Since I'm renting a server, I can't exactly add RAM straight to it...
Although I could get upgraded to a server with an additional 256MB of RAM in it for $10/month...
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
adelikat wrote:
Only $10? Upgrading shouldn't be a problem then, if $10 is too much for you, talk to me about it.
$10 a month.
Well if the site can generate at least $10 a month, via ads/donations or whatever, then yes, I can go for that.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
