Please consider donating to help with the server renewal costs. See the forum thread here.
New to the server? Click here for the main website.

Excessive and Unnecessary Map Explorations

Hi friendly community!

In the past few days, several players had issues with their inventory disappearing, floor blocks were missing, and other glitches and issues. This Forum was also down. The Culprit? Full disk space! But the full disk space was due to excessive map explorations (aka spider-legging the map overviewer).

Have you seen our map overviewer? Every time we explore new land, that is additional data that our server must save, not just for the sake of the overviewer, but because it records the location and ID of each loaded block, which is used for rollbacks and griefing protection. Large and unnecessarily explored maps increase the data requirements exponentially.

Please stop doing that :)

To our valued new influx of players, keep having fun! But understand the implications of traveling out to stinkin -34,000!! Please reserve such massive exploration for your single player game. It's not cool to do that to here, costs extra money to host, extra time for the server to save the world, and extra time to research what the heck happened and fix the problem so that we can have a functioning server for the other players.

Comments

  • This problem won't go way. Not until we get more hardware. Which probably won't happen over night.

    My suggestion: institute an exploration limit. No one can travel more than 3k blocks from spawn. You risk having your house deleted if you do. Next, open the map in MCEdit and remove any and all blocks that fall outside of this limit.

    I'm not sure what a good limit is. 3k? 5k?
  • That's unfortunate, since the one thing I really liked doing in Minecraft was seeing all the weird and wonderful landscapes it could come up with.

    Oh well, That entire North Western spur is me - out to around -32k,-18k or so.
    If you can selective chop data from the system you can delete everything that's out there.

    (exits stage left)
  • I apologise for my recent explorations, although I haven't gone the furthest, I have been trying to fill in the map more. One problem we have been having with this map is the over-abundance of desert and savanna, and the conspicuous lack of jungle, I was attempting to find one closer than the only one so far located (http://minecraft.sitosis.com/map/#/-34839/64/-17274/-2/0/0), but I was unsuccessful.
  • The closest snowy biome is just over 6k blocks from spawn. I am interested in keeping to the snowy clime that is there.

    Any ideas?
  • Could we delete the land in between and use travel in the nether as a preference?
  • Well poop. So much for Manifest Destiny II: The Walkening.

    Just out of curiosity, how much of a donation bump would it take to get enough for a hardwate upgrade?
  • Oh no! I feel really bad. there is definitely stuff that we can do to keep the size manageable and let the game still be fun for our travelers (like yexploringblacksmithe and dmcgibbon). I don't want to restrict you since not everyone plays the same!

    costs to upgrade: TBD. also, im hesitant to delete land without a thorough investimagation to make sure we're not scrapping someone's work.

    I wanted to put it out there and im glad to know more about players' motivations :> glad to hear no one is afflicted with restless leg syndrome
  • I'm sure we can grandfather people in. We don't have to delete everything beyond 3k or whatever we decide. It's just a general guideline to keep things under control.
  • It's no big deal on my part. MDII is on hold until Wild West One is further along, and also Innsmouth is very alluring.

    I do like to explore, but I wouldn't want to cause problems by doing so. With a general idea of upgrade cost, i (and perhaps other stalwart explorers) could maybe budget a bit extra towards that goal.

    Something to think about, anyway.

    Also, Thanks to all ya'll Admins for your hard work!!!
  • Donation Drive for more hardware is a good option, Our player base went up like a thousand percent so it seems pretty logical.

    MCEdit might be an option, probably needs research into thing like what happens to inventories, explored map you want to keep etc.

    Also there is this for the future http://minecraft.gamepedia.com/World_border
  • If you deleted chunks, wouldn't they be re-rendered when someone else came in range, and would they be the same? I'm not sure how much of the terrain depends on the seed and how much has a random tweak added after
  • Actually, thinking about this; I think it's more a result of the increased number of people on the server - although my wanderings won't have helped. But remembering that the world is 3D, so the more people we have digging and exploring underground, the more data the map needs to hold. And since we're all trying to keep separate land claims, that's a lot more data.

    What's the spec of our current server (I'm in IT, so don't spare the gory details :) ) ?
  • Does 1.8 use more space for chunk storage? Because I don't think we've explored as much as on the previous map. Or is it because the previous map is still on disk? If so, could the latter be tarred up and removed?

    I'm just outside the 3k limit, and I would be discouraged to lose my place.
  • It's all about the number of chunks the server has to keep in memory and on disk. Having a bunch of people spread out all over the place puts a strain on the hardware.

    I've heard of some servers using world border. They gradually increase the size as time goes on to ensure the server can handle it.

    Personally, I think we should delete some of the really big offenders. Mainly the paths that go way out to nowhere and probably won't be used by anyone. It's pretty easy to pick out those spider legs.
  • The number of chunks in memory is governed by the number of people on the server at any one time, surely? MC should unload chunks when there's no one around in that vicinity. That said, I've noticed a sort of stutterring in the sky, especially at night, so maybe our server isn't up to the load of new players. Mapped areas should only contribute to disk space once someone's caused them to be rendered (guilty!).

    I'm happy enough for my big long trail to be deleted (though if you keep the end point where my actual base is, it'd be appreciated :) ).

    "The way out there has been forgotten and passed into vague memories"
  • I plead guilty to exploring that big U-shape to the north-west (http://mc.sitosis.com/map/#/-9844/64/-11129/-7/0/0). I'm sorry, I didn't think about server space. You can delete that, there's nothing really interesting there.
  • There's a bunch of speculation and questions above, so, I'll try to address them here in a random order. Probably will forget something.

    * It isn't about the server memory right now, it's the disk space. More of the map people explore the more map data gets generated and stored on the disk. Nothing to do with the map overviewer on the website, that just displays the data that MC generated.

    * Server memory is relevant when thinking about concurrent players - how much of active map data does the server have to deal with ATM; how many chunks does it have to load into memory. The more spread out the online players are, the more chunks it takes (people close by will likely be on the same or neighboring chunk). So far I haven't seen any major lag even with like 15 people online. Fingers crossed, but that seems to be OK for the time being.

    * The disk doesn't just hold the active map, but also 3 days worth of backups from every 4 hours. That's for griefing or some potential map corruption protection. I could make something like only one backup per day, but that means if we need to roll back, we would lose minimum of 24h of work, instead of 4h.

    * I don't know if the map format changed, but we are already at > 1 GB per compressed map file. During the past 3 years we got there (if ever) after many months, when we were ready to generate a new map.

    * I don't want to restrict exploration, but going to 30k seems a bit excessive. Chances are (not certain, of course) the biome you're looking for is not that far and you just missed it.

    * Deleting chunks and then having them regenerated will produce the same terrain. It's based on the map seed. I've previously generated old Sitosis maps on SP just from the map seed as a test. Everything looked the same. I'm not sure if it also applies to the locations of the dungeons and such, though.

    * I'm open to the idea of donations to increase the disk space. We are looking into that ATM. However, it will raise the cost of the service renewal in February, as well. The player donations only cover some of it and the rest is coming from my pocket. This year I seriously considered lowering the server specs to save money, but finally decided not to. Thankfully, some of the players who recently joined us have already donated, which helped to offset my own expenses a lot (thank you guys!). To put it simply - how much money we are able to raise during the annual donation drive in February, will decide how much RAM and disk space we can have for the coming year.

    * We are currently running a VPS with 3 GB of RAM and 50 GB of disk space. The server runs MC and hosts the website.

    ...

    I need a snack and a nap now :P
  • asum2: The transparency is incredibly welcome! Thanks for putting in the time to keep everything clear.

    dmcgibbon: asum cleared up most of the details, but it's also worth mentioning that digging underground does not affect terrain generation; Minecraft generates chunks as vertical pillars 16 x 16 x 256 blocks at a time. As you move north, south, east or west in to unexplored space, new chunks get made, but not as you move up or down. There's a good explanation of it here: http://minecraft.gamepedia.com/Chunk

    The game renders the world in 16x16x16 cubes, but that's a RAM issue (specifically for your machine, rather than the server) and not related to storage.
  • OK, since it looks like the bulk of this has been caused by my explorational tendencies, you could start by erasing that whole isolated section of exploration from around -21k9, -10k up to -34k4, -17k8
  • edited September 2014
    I deleted everything beyond about 9600 blocks in each direction in the overworld (.mca files with either coordinate >= 19 or =< -19, for those who know what I'm referring to).

    This was urgent as the disk space was critical. If anyone was affected by this, contact an admin and we'll help you (TP you, compensate any lost materials etc).
  • Nice. The server feels pretty stable now.

    Thanks asum and frag for getting this resolved. I'm off to donate. :P
  • I agree with the huge exploration runs. I'm not a minecraft pro but anything to cut down the lag and instability of the server would be great. And I feel like I've just started on this map and the map is blowing up in size at a huge rate. I would appreciate going for quality over quantity.
    That being said, I couldn't login to the server today. and now it seems that it is down. I'm assuming this is because the map is being chopped down right now...?
  • Looks OK ATM. It was down before - the map pruning took seconds, but re-generating the overviewer map took some time and I shut down MC during that as it would have been extremely laggy anyway.
  • It also might be worth looking into some other storage mechanism for backups, like s3 or the like. The thought is that you will probably get more GB/$ out of a system dedicated to storage than you would by purchasing it all through a VPS that scales simultaneously out in disk space/RAM/residency percentage

Leave a Comment

bolditalicunderlinestrikecodeimageurlquotespoiler
Drop image/file