[Wishko]

С самого запуска "Сезона открытий" PvP-событие Битва за Ясеневый лес испытывает множество проблем от самых малых вроде переброса на другие слои посреди боя до крупных и неприятных вроде полного зависания. Старший продюсер WoW Classic Джош Гринфилд опубликовал на форуме огромную заметку, в которой детально рассказал об этих неполадках и их причинах, а также поведал, как они будут решены в ближайшее время. Подготовили пересказ.

Проблемы

• Главной целью Битвы за Ясеневый лес было "заманить" игроков в PvP, предложив в виде награды репутацию с фракциями Ущелья Песни Войны. Само по себе это событие PvE-направленности, однако после и до него в виду появления большого числа игроков разгораются нешуточные PvP-баталии, благо сам Ясеневый лес к этому очень располагает. Задумка удалась и в локации постоянно кипит PvP.
• Само по себе событие стабильное и было найдено несколько незначительных проблем, которые были быстро решены в первые дни. Однако это касается только случаев, когда событие проводится на игровом мире с одним слоем, а когда слоев становится больше, то возникают многочисленные конфликты с различными системами вроде автоматического балансировщика фракций и исправлениями. Это проблемы проявляют себя как неожиданный выброс с текущего слоя посреди боя, сброс счетчика прогресса до начала боя до низких значений, частичное или полное зависание события. Далее в деталях рассматривается каждый из этих моментов.

Выброс со слоя

• В WoW Classic действует система слоев, которая в автоматическом режиме запускает копии целого континента по мере того, как на игровой мир входит все больше игроков, чтобы решить затруднения со стабильностью игры и заполненностью локаций, и сворачивает их по мере снижения числа пользователей. В обычных условиях система старается не перемещать игрока между слоями, особенно если он находится в группе или в бою, чтобы не вредить игровому процессу и не перебрасывать его постоянно туда-сюда, однако, когда число пользователей на слое достигает критического значения, это все же может происходить.
• В Битве за Ясеневый лес игроки часто собирают огромные рейды (в том числе с помощью внеигровых инструментов), поэтому иногда слои заполняются до абсолютного максимума и система, игнорируя все ограничения, в том числе на присутствие в группе, переносит некоторых персонажей на новые слои, чтобы избежать серьезных проблем с производительностью и аварийным завершением работы слоя.
• Ранее слои не сворачивались сами и пропадали только с перезагрузкой игровых миров, поэтому были нередкими случаи, когда их открывалось много в пиковые периоды, а затем после выхода части игроков из игры небольшое число пользователей оставалось одиноко разбросанным по этим слоям, из-за чего мир казался пустым. Для решения этой проблемы и оживления мира в "Сезоне открытий" разработчики позволили слоям автоматически закрываться и перемещать игроков на оставшиеся. Это привело к аналогичной проблеме – при закрытии слоя некоторых игроков вопреки их желанию также может перенести на другой слой в самом разгаре игрового процесса.
• Таким образом, выброс с текущего слоя мог происходить как из-за обилия, так и недостатка игроков на нем, а это в свою очередь могло привести к неприятным последствиям, особенно в Битве за Ясеневый лес.
• Вчера разработчики выпустили срочное исправление, которое сильнее ограничило систему и теперь она старается жестче учитывать присутствие игрока в группе и в бою и не перемещать его на другой слой. Однако это все равно может происходить, пусть и гораздо реже, в особо критичных случаях, когда возникает угроза производительности игры. А завтра разработчики насовсем отключат автоматическое сворачивание слоев и потому в часы низкой загрузки мир может показаться пустым для некоторых игроков, но это вынужденная жертва во благо.

Счетчик прогресса

• Иногда счетчик прогресса во время подготовки к событию полностью или частично сбрасывается, например с 70-80% до 20-30%. Это происходит из-за того, что он динамичный, пропорциональный и учитывает общий на весь сервер прогресс со всех активных слоев. Когда на одном из слоев событие завершается, то масса игроков с него начинает учитываться счетчиком и потому требуется совершить уже много больше (+25% от предыдущего значения за каждый дополнительный слой) PvP-побед для запуска события, а для тех, кто уже некоторое время трудится над заполнением шкалы набранный процент заметно снижается.
• Это пусть выглядит странно, все же является задуманным поведением. Разработчики хотели, чтобы Битва за Ясеневый лес начиналась на всех слоях одновременно, а игроки не перескакивали со слоя на слой вручную в надежде поучаствовать в событии несколько раз. Эта идея во многом сработала, однако принесла немало неудобств самым активным слоям, поскольку бой на них завершается скорее и они быстрее приступают к заполнению счетчика, несколько раз наблюдая падение процента, когда сражения завершаются на более медлительных слоях, что неприятно.
• Для решения этой проблемы вчера они убрали снижение % и сделали так, чтобы прогресс позднее завершивших битву слоев попросту поднимался до текущего актуального.

Зависание события

• Исправление со счетчиком прогресса привело к появлению новой проблемы, из-за которой событие начало частично или полностью зависать. Если на одном или нескольких слоях событие не успевало завершиться до того, как другие слои завершили заполнение % прогресса, то система пыталась заново запустить уже идущее на них событие. Это приводило к тому, что часть прогресса уже идущего боя терялась, а в некоторых случаях события могли намертво зациклиться и игроки попросту не имели возможности его завершить до ручного сброса/перезагрузки разработчиками.

Что дальше

• Помимо отключения автоматического сворачивания слоев завтра, сегодня разработчики выпустят еще одно срочное исправление, благодаря которому число необходимых для запуска события побед будет сильно увеличено. Это замедлит частоту проведения Битвы за Ясеневый лес в пиковые моменты (они будут проводиться раз в пару часов) и значительно замедлит частоту вне пиковых моментов. Благодаря этому сильно снизится вероятность ситуаций, когда к моменту заполнения общего прогресса на сервере будет оставаться какой-либо слой, где событие еще не завершилось, и, как следствие, будет происходить его нарушение. Изменение не самое идеальное, но разработчики хотят как можно скорее стабилизировать ситуацию. В будущем вся система скорее всего переработана, но это слишком крупная задача, чтобы выпускать ее срочным исправлением.
• Разработчики понимают, что отсутствие PTR для "Сезона открытий" позволило сохранить интригу и сделать игру интереснее, но также это по сути привело к появлению подобных проблем, которые можно было бы обнаружить и исправить во время публичного тестирования или стресс-тестов. У всего есть свои минусы.

Разработчики благодарят сообщество за игру в "Сезоне открытий" и обещают, что примут во внимание проблемы текущего этапа и решат их к следующим, которые будут еще увлекательнее.

Update on the Battle for Ashenvale and Layers - 12/14/2023

Sometimes in game development you find yourselves in situations where things don’t go according to the design or plan you have in mind.  In the past when we’ve hit such situations, we’ve done our best to remain transparent and take some time to explain what our intent was, what went awry, and what our next steps are. We’d very much like to carry that forward with Season of Discovery, where it’s especially important to be open and honest. Season of Discovery is, by its nature, highly experimental and we were bound to hit some snags along the way but we think it’s a best practice to simply explain what happened and learn from it.

So, we are going to really dig in deep into what has been happening with the Battle for Ashenvale, but word of warning, this is going to be an extremely long, extremely dense post. I’ll be taking time to explain all of the major systems at play so if you aren’t in for the technical details, you may want to skip to the end for info on what exactly we are doing now.

The Intent

The intent of the Battle for Ashenvale was fairly simple; use a fairly compelling carrot on a stick (Warsong Gulch reputation and a mount) to get people into Ashenvale so they could fight. In that respect, it was a success. As Ashenvale is a hotly contested zone that houses several quest hubs as well as our “end-game” raid for the level 25 level band, some conflict was assured here, but we really wanted Ashenvale to be a non-stop hotbed for PvP. The event itself has a decidedly PvE slant, which is fine, but in the lead up to and aftermath of each battle, there is essentially non-stop PvP at all times throughout the zone and we couldn’t be more pleased by how that aspect of this has played out.

How The Battle for Ashenvale works under the hood

Looking at the Battle itself though, it might help to prime the rest of this post with a look at how it functions under the hood.

• The Battle for Ashenvale cycle starts with a lead-up phase.
• During this lead-up, there is a counter that runs behind the scenes and adds up how many PvP and PvE kills occur in the zone.
• When the counter reaches a certain number, the battle begins.
• This is represented on-screen with a percentage counter UI element in the top-middle of your HUD.
• This element is replaced with an objective tracker showing the progress of the battle when it starts.
• Once the battle begins 3 Lieutenants spawn for each faction at various points on the map; Keepers of the Grove for the Alliance, and Blademasters for the Horde, as well as a general for each faction. For Horde this is a Farseer, for Alliance this is a Priestess of the Moon, and they are both invulnerable and do massive damage until all of their lieutenants are killed. The objectives are simply to kill the opposing lieutenants and then kill the enemy general. Once this is done, the battle ends and you return to the lead-up phase and the cycle repeats.

What Went Wrong

On a realm with a single layer, there are very few points of failure with this system. We had a few minor issues in the very early days of Season of Discovery with the base functionality of this system and for the most part those were quickly resolved. The issues we are seeing now are the result of a conflict between multiple different systems that are meant to load balance server populations as well as unintended interactions between some fixes we made earlier this week with the battle.

To help organize let’s break these issues down into the main symptoms and then we’ll dive into the root causes:

• Being removed from your current layer mid-battle.
• Progress Counter Resetting back to 0 or low numbers during the lead-up phase.
• Battles becoming stalled and the objective tracker stops responding.

Being removed from your current layer mid-battle

Layering is a system that we’ve had in place since the launch of WoW Classic in 2019 to allow many times more players onto a single realm than would otherwise be permissible without severe performance degradation. Layering essentially creates fully featured copies of the entirety of Eastern Kingdoms and Kalimdor. The number of layers on a realm depends on the peak number of players logged in recently.  Historically, when realms start up during weekly maintenance we start with a nominal number of layers—generally only a few. As we move closer to peak play times in your region however, additional layers would spin up dynamically as more and more people log in and existing layers fill.

The layer manager is a system that controls layers, spins them up as needed, and decides which layer players logging in will be assigned to. The layer manager will try not to move you to a new layer unless certain thresholds are crossed. In normal times outside of major events and launch windows, this almost never happens.

When the world begins to fill on an individual layer and that layer begins reaching a critical mass however, the system may at times try to load balance the population on that individual layer to an existing layer or a brand new layer if all others are full, which at times means moving people onto a relatively empty layer. The layer manager system does have some safeguards to prevent you from too many jarring layering incidents, such as preferring NOT to move people that are in a group with each other, or that are in combat. Ideally, what will happen when a group is formed is that a re-evaluation will happen when people leave and join, and the layer manager will attempt to get everyone onto the same layer and keep them there until they all log out.

The snag we run into with the Battle of Ashenvale however, is that often times a great many people are joining groups, often times using external tools like discord to coordinate. When a player leader organizes several full raids to all congregate on one layer, that layer may hit a threshold and start trying to not add more people to that layer. For a time it will continue to allow more people to join to not break up groups, but there is a point when it reaches its absolute maximum and on the next evaluation the manager will forcibly move people off of the layer to balance the load and prevent a crash or other severe performance drops. This means that even players that were grouped and would normally be protected from being moved because they are grouped may also be moved as an emergency measure. This has been one cause for players unceremoniously being booted to a different layer; the layer simply got too full and stopped caring whether you were grouped or in combat or not.

Going back to what I said a moment ago about layers spinning up as we move into peak time, one key detail is that in the past, layers would never spin down or “retire” unless we restarted the realms completely. This means that at peak times, there may be 5, 6, 7 or even 10 (or more) layers going at once on the largest realms. At off-peak times however there would still be that same high number of layers and several days after weekly restarts, there would be so many layers going that late at night or early morning the world would feel very empty. This means even if several thousand people—what would normally be a “large” realm in earlier versions of WoW—are logged in at 8:00 AM but there are 10 layers going from the previous day’s peak, each layer only has several hundred people (when it could handle several thousand) and it could feel pretty lonely.

For Season of Discovery, we decided to try something new and allow layers to automatically retire when populations dropped to “collapse” the number of total layers and make the world feel more active and lively at all hours of the day and night. We hoped to avoid those “dead” periods where the realm felt more empty than it actually was.

The unfortunate side effect of this is that every time we would retire a layer, it would suddenly  cause players to change layers as the layer manager re-balanced and redistributed folks. We’ve never engaged this system before because we knew this would feel jarring, and in a game mode like Hardcore for example, being unexpectedly moved to a new layer when yours is retired could be deadly! But, like many other aspects of Season of Discovery, this was a risk we consciously took to see if we could improve the experience for those that prefer to play at off-peak times.

As you’ve likely deduced buy now, both of these extremes; too many and too few people attempting to play could cause players to suddenly be moved. This included people in groups, people trying to join groups, as well as people actively engaged in battle in Ashenvale.

As of yesterday, we have already made a change that should cause the layer manager to try harder to prevent players from layering if they are a) in a group, and b) in combat. This is not to say it will never happen. When a layer reaches a certain level, we have to split people up or it will crash. That is unfortunate but currently it’s unavoidable.

The next change we are about to make starting tomorrow is to stop retiring layers when the population dips down during offpeak. This means that you won’t be suddenly moved because your layer is being retired, but it also does mean that if you are playing very late at night or early in the morning, the world may feel sparsely populated.

Progress resetting during the ramp-up period

The next major issue involves the ramp-up period percentage counter suddenly dropping from a high percentage to a low one. The cause for this is simple; the progress tracker represents an aggregate of progress across every participating layer. This means that when you finish a battle and start making progress and other layers also finish their battles (or new layers spin up), they all join the “pool” of layers contributing to the total percentage progress.

So, if your battle ends and your layer managed to get back to to 20% towards the next battle, and then another layer entered the ramp-up phase, they would start at 0% which would drop the total progress across all layers down several percentage points as the “pool” and thus the total required number of kills just got larger. This was technically intended behavior, but it was very confusing. You’d be questing, pvping, or gathering in Ashenvale when you’d see the counter go from 70 or 80 percent suddenly down to 20 or 30 percent as other layers had their battles wrap up and they jumped into the pool.

The original intent of this was to make it so that all battles started at the same time across every layer on a realm. This was to try and prevent players from hopping layer to layer to attempt to do the event multiple times back-to-back and place additional load on the layer manager. While that was somewhat successful, the trade off was just too much confusion as the most diligent layers who finished their battle early would see their progress drop potentially multiple times during the following ramp-up phase.

To help with this, we made an adjustment yesterday to never allow the aggregated progress %  to drop. New layers who joined the pool of layers that were in the ramp-up phase would just have their total progress updated to match whatever current % progress value is displayed. In a vacuum this change makes sense and seems like it would help prevent that confusion, but this is really the change that led to the most pronounced issues.

Battles becoming stalled and objective trackers not responding

The change mentioned above created a situation where multiple layers could finish their battle, enter ramp-up again and start making progress very rapidly towards their next battle. However if one or more layers did not finish their battle before the other layers in the ramp-up period finished said ramp up period, it would cause any in-progress battles on that realm that were lagging behind to completely break. This is because the system that controls when a new battle starts would see that the ramp-up was finished and send the signal to all layers to start a new battle, including those with a battle already in progress.

In addition to breaking any in-progress battle this would likely also cause some layers on your realm to enter a veritable death-loop of never being able to finish a battle before a new one started, breaking their current battle all over again. This was even further exacerbated when a layer automatically retired itself, because then the total number of kills needed would go down and the progress needed for the next battle would suddenly jump up and make it even more likely that a new battle would start when some layers already had one going.

There’s one last detail to mention before discussing what the path forward is; and that is how additional layers impact the total progress towards the next battle. Using very fake numbers, lets say that if you have a single layer, you need 1000 creature or player kills to trigger a battle. The way we had it set up was that if a second layer spun up, you’d need 25% more kills, so it would go to 1250 needed total between both layers. A third layer would bring the total needed across all layers to 1562 (1250*1.25), a fourth made that number go to 1925, and so on. When we originally designed the system, we initially made each layer require more kills per layer on a 1:1 basis, so using the numbers above as an example, with 5 layers you’d need 5000 kills (5 layers x 1000 kills needed per layer). We decided not to do this later in development because we had been operating under the assumption that layers never really spun down outside of planned restarts, thus increasing the kill requirement on a 1:1 basis would cause battles to take an extremely long time to trigger during off-peak times because there were many layers and relatively few players around to make progress during the ramp-up period.

All of these many words brings us to the crux of it; the system was not really designed with the idea that a) we’d ever be retiring layers other than during restarts and b) that we would get to a point where a battle could still be going on a layer when the next battle began.  This is also an example of a system that could have really benefitted from a proper PTR stress test and we are going to keep that in mind for the future.

TLDR and the path forward

The next set of changes we have on deck are:

• With a hotfix tomorrow we will no longer be dynamically retiring layers. This means that late at night or early in the morning there will be more layers than are needed and the world may feel more empty.
• With a hotfix tonight we are increasing the number of kills that are needed for each layer that exists to move the overall realm percentage up.

Within an hour or so from the time this is posted, most layers should have “healed” themselves, but we will be watching it closely.
Both of the changes together do also mean that battles will spin up FAR slower during offpeak times and noticeably slower during peak times. The trade-off is that it’s now far less likely that one layer will lag far enough behind to break if the percentage counter fills up before their battle completes. We believe it should now be a few hours between battles, at least, even during peak. We need this buffer to prevent these stall-outs from happening.

This solution is not perfect though and we may consider a further redesign of this system down the line, but that’s likely too large of a change for a hotfix right now. The priority now is to stabilize things with this most recent set of changes.

Thanks so much if you read this entire post. I understand if you have questions about this as this is a lot to grok. Suffice it to say that the ultimate issue here is that we had a collision of a few different game design and server engineering systems that were both meant to enhance the player experience, but ultimately ended up damaging said experience when they clashed. We took a chance by not having a PTR. Some things in Season of Discovery were a success due to the lack of a PTR, but this was likely less successful.
Lastly, thank you so much for playing Season of Discovery with us. This season has been a true labor of love for the WoW Classic Team and while its certainly been interesting to support, we can’t wait to take the lessons we’ve learned forward to make future phases even more awesome![Источник]

[xALEKSx]

Господи, зачем мне как игроку всё это знать?

[Этрей]

"Это замедлит частоту проведения Битвы за Ясеневый лес в пиковые моменты (они будут проводиться раз в пару часов) и значительно замедлит частоту вне пиковых моментов. " - количество репутации за событие конечно же повышено не будет (чтобы компенсировать более редкое его проведение) 

[ihalf21]

Т.е. тем, кто не успел нафармить репу теперь эту будет сделать кратно сложнее? Хороший ход) Ладно хоть маунт уже на френдли взять можно

[Starui]

Эх) а как круто было когда анонсировали фронты в БФА