Changes

12,680 bytes added ,  06:18, 17 September 2021
→‎System Titles: system stability intensifies in the kernel
Line 80: Line 80:  
* [[HID_services#Firmware_update|ControllerFirmware]]: updated "FirmwareInfo.csv", "TouchScreenFirmwareInfo.csv", "ukyosakyo_ep2_ota.bin". Added "FTS_33000510.fts256" and "FTS_98000004.ftb".
 
* [[HID_services#Firmware_update|ControllerFirmware]]: updated "FirmwareInfo.csv", "TouchScreenFirmwareInfo.csv", "ukyosakyo_ep2_ota.bin". Added "FTS_33000510.fts256" and "FTS_98000004.ftb".
 
* Various applet UI/gfx data was updated.
 
* Various applet UI/gfx data was updated.
 +
 +
===BootImagePackage===
 +
All files in RomFS were updated.
 +
 +
====Kernel====
 +
* Compiler upgrade to LLVM 11.1.
 +
** Most notably, certain code now emits "ands" rather than "and; and; tst" + "bfxil" patterns are more commonly used.
 +
* C++ language upgrade to C++20 (or C++17, but less likely).
 +
** Slab heaps/object containers are now constinit, no longer constructed during .init_array.
 +
* Initialize0 changes:
 +
** KernelLdr now sends back the initial process binary address in state.
 +
** Initialize0 now uses a helper function for selecting random virtual regions, and now verifies KInitialPageTable->IsFree() before selecting regions.
 +
** Initialize0 now verifies that the initial process binary address is in correct place (pool partition), and sets global=initial address.
 +
*** This global is now used where GetInitialProcessBinaryAddress() was used previously.
 +
* KPort limit increased to 0x180 from 0x100.
 +
* SvcMapDeviceAddressSpace() was removed.
 +
** Userland only ever used SvcMapDeviceAddressSpaceByForce and SvcMapDeviceAddressSpaceAligned, so this doesn't break any official software.
 +
** Map only allowed one page table to be allocated, only partially mapping the desired range if more than one would be used.
 +
*** Similarly, it returned an output mapped size so a caller could continue mapping partially until the whole range was done, one page table at a time.
 +
** All parameters required to implement partial mapping have been removed, correspondingly.
 +
*** No out_mapped_size, out_page_table_count, pt_limit parameters to any KDevicePageTable mapping functions any more.
 +
*** UnlockForDevicePageTablePartialMap no longer takes in a partially-mapped size.
 +
**** This code is simplified, since it is no longer possible to have a partially-mapped size other than zero.
 +
* The two new kernel objects/four new SVCs added in 11.0.0 are finally present/instantiated.
 +
** Prototypes for the new SVCs:
 +
*** 0x39: Result CreateIoPool(Handle *out_handle, uint32_t which);
 +
*** 0x3A: Result CreateIoRegion(Handle *out_handle, Handle io_pool, PhysicalAddress physical_address, size_t size, MemoryMapping mapping, MemoryPermission perm);
 +
**** MemoryMapping is a new enum, 0 = IO, 1 = Uncached memory, 2 = Normal Memory.
 +
*** 0x46: Result MapIoRegion(Handle region_handle, uintptr_t address, size_t size, MemoryPermission perm);
 +
*** 0x47: Result UnmapIoRegion(Handle region_handle, uintptr_t address, size_t size);
 +
** These SVCs conceptually allow creating an object for mapping in certain physical address ranges at user-specified virtual addresses, without having to include the ranges in npdm/kip capabilities ahead of time.
 +
*** The only allowed id right now is 0, which corresponds to PCIE_A2 (physical address range 0x12000000-0x1FFFFFFF).
 +
* KWritableEvent was deleted.
 +
** Class tokens for types after KWritableEvent have been adjusted downwards to compensate (see mesosphere for class token generation algorithm).
 +
** Handles which were previously returned to KWritableEvent are now returned directly to KEvent.
 +
** KEvent has new boolean member to track whether the readable event has been destroyed.
 +
** KReadableEvent::Initialize now opens reference to the parent event, rather than inline in KEvent::Initialize.
 +
* KReadableEvent::Signal/Clear are no longer virtual functions.
 +
* KAutoObject no longer has virtual destructor.
 +
** This mostly means that base/deleting destructor no longer occur inside vtables.
 +
* New memory state (0x16/0x2016) "Coverage", currently not exposed via any SVCs.
 +
* A number of functions are now devirtualized when possible, indicating either the virtual functions (or the classes) were marked final.
 +
** KProcess::GetId()
 +
** KThread::GetId()
 +
** KSessionRequest::Finalize()
 +
** KInitialPageAllocator::Free()
 +
* KConditionVariable::WaitForAddress/KConditionalVariable::SignalToAddress are now static
 +
* KMemoryManager now operates on physical addresses instead of virtual addresses.
 +
** KMemoryManager::Initialize now iterates the physical tree rather than the virtual tree, when finding pool regions.
 +
** KMemoryManager::Allocate now returns a physical address instead of a virtual address.
 +
** KMemoryManager::Close now takes in a physical address instead of a virtual address.
 +
** KPageHeap now operates on physical addresses rather than virtual addresses.
 +
*** New member stores the linear virtual address for the physical range the heap operates on.
 +
** KBlockInfo now size 0x10 instead of 0x20.
 +
*** Before: struct { KBlockInfo *prev; KBlockInfo *next; size_t num_pages; KVirtualAddress address; }
 +
*** Now:    struct { KBlockInfo *next; u32 phys_address_page; u32 num_pages; };
 +
*** phys_address_page is a KPhysicalAddress / PageSize, to ensure it fits in u32.
 +
** This halves memory requirements for KBlockInfos system-wide, effectively doubling the KPageGroup capacity.
 +
** Memory range helper object used by page table functions also now operates on physical addresses.
 +
* KPageTableBase no longer contains virtual memory region cache.
 +
** This was only used in KPageTableBase::MapPageGroupImpl to do ABORT_UNLESS(IsHeapVirtualAddress(...));
 +
** This abort is no longer present, likely because page groups are now physical blocks rather than virtual ones.
 +
* KPageTable::Unmap now incrementally frees pages, rather than freeing them all at once.
 +
* KHandleTableEntryInfo no longer stores object class token, KHandleTable::Add/Register no longer takes class token as argument.
 +
** This reduces KHandleTable size (and thus KProcess size) by 0x800.
 +
* The following types no longer have (unused) slab heaps:
 +
** KClientSession, KLightClientSession, KLightServerSession
 +
* A complete re-work/unification was done for the various kinds of thread waiting operations in the kernel.
 +
** Previously, there were a number of different ways to initiate and end waits, handled manually in each location by invoking thread->SetState(...);
 +
** Now, all waits use a common interface based around thread queue objects with virtual functions for the three kinds of supported waits.
 +
** Supported queue virtual functions:
 +
*** void NotifyAvailable(KThread *waiting_thread, KSynchronizationObject *signaled_object, Result wait_result);
 +
**** This is supported only by the queue used by WaitSynchronization, and sets the signaled object/sync'd index for the thread.
 +
*** void EndWait(KThread *waiting_thread, Result wait_result);
 +
**** This conceptually ends a wait "normally", usually without additional cleanup. This is exclusively called by kernel-handled wait codepaths.
 +
*** void CancelWait(KThread *waiting_thread, Result wait_result, bool cancel_timer_task);
 +
**** This conceptually ends a wait "by force", interrupting it regardless of normal wait completion.
 +
***** This usually involves extra cleanup.
 +
**** This is invoked by e.g. CancelSynchronization, thread termination, thread finalize, etc.
 +
** KThread::SetState is likely private now; it is called only by internal KThread functions.
 +
** KLightSession was notably reworked substantially to take advantage of the new unified wait semantics.
 +
** KSynchronizationObject::DumpWaiters() no longer exists.
 +
* KDebug is now substantially more careful about management of its process pointer.
 +
** New field "is_attached" explicitly tracks whether the KDebug is attached (instead of m_process != nullptr).
 +
** m_process replaced by a new helper object containing KProcess * and reference count.
 +
*** Code which previously used the process field now opens/closes references on m_process_holder.
 +
*** When m_process_holder's reference count hits zero, the process is closed and m_pointer is set to 1 instead of nullptr.
 +
** This makes the way KDebug treats the attached process's reference count much more correct/consistent.
 +
*** In particular, "the KDebug is attached" now counts as one reference to the KProcess, no matter what the KDebug is doing with it.
 +
* Resource management has changed substantially.
 +
** Dynamic slabheaps no longer contain a pointer to an associated page allocator; instead, a page allocator is now passed as an argument to .Allocate().
 +
** There are new helper objects which contain a dynamic slab heap pointer and a page allocator pointer, used for conveniently referencing a slab/page resource pair.
 +
*** There are now helper objects differentiating between KBlockInfo and KPageTableManager allocators for system/application context.
 +
*** KPageTable now has a helper function to allocate a page from the KPageTableManager helper object.
 +
** Instead of allocating all unused pages to the page table page heap, all but 70 pages are allocated to the page table page heap.
 +
** When a new flag from the secure monitor is zero (this is always the case on retail), new logic for "dynamic resource expansion" is enabled.
 +
*** In particular, this causes the helper objects for the system memory blocks, block infos, and page table managers to be set to the dynamic page allocator.
 +
**** Thus, the last 70 unused pages are dynamically allocated to (system memory blocks, system block infos, system page table pages) on a first-come basis as the system uses these resources.
 +
**** NOTE: There is no "free"-ing of these resources back to the unused page heap, once they've been allocated to a specific slab they will remain in that slab until reboot.
 +
*** In addition, when allocating a KSession, KEvent, or KLightSession from the system resource limit fails, an object of the desired type will be allocated from the otherwise unused "gaps" in the slab region.
 +
**** When allocating a KSession dynamically from gap-space, kernel also allocates two KSessionRequests from gap-space and frees them to the KSessionRequest slabheap.
 +
**** Allocator for this uses an intrusive red black tree on in-place nodes in the gaps.
 +
* Minor changes to the atomic operations for slab heaps.
 +
*** Slab heap initialization now panics if L1 cache is direct-mapped (1-way associative).
 +
*** Slab heap allocation no longer issues a clrex on failure.
 +
*** For types supporting the new dynamic allocation, Slab heap free now checks that the object lives within the slab region instead of panicking when object is not within slab extents.
 +
* The interrupt task manager thread no longer exists, functionality has been rolled into KScheduler.
 +
** KScheduler now has member pointer to the interrupt task manager for the current core.
 +
** KScheduler::EnableScheduling now sends scheduler interrupt unconditionally, regardless of disable count.
 +
** KScheduler::RescheduleCurrentCore now uses double-checked-locking-esque strategy for checking scheduling necessity around interrupt disables.
 +
** KScheduler::Schedule now processes interrupt tasks directly (calls a KInterruptTaskManager member function) rather than setting the interrupt task thread to runnable.
 +
* KInterruptEventTask no longer contains a KLightLock member.
 +
** KSchedulerLock is used in places where interrupt events were locked previously.
 +
* KSchedulerInterruptTask is now a KInterruptHandler, rather than a KInterruptTask (no ::DoTask() implementation, any more).
 +
* KProcess::AddSharedMemory now sets all fields in one block/scope, rather than two.
 +
* KWorkerTaskManager::Initialize now hardcodes id=0 instead of taking it as a parameter.
 +
** KWorkerTaskManager no longer has "id"/"active" fields.
 +
* Certain unnecessary conditionals were optimized in KServerSession::SendReply/ReceiveMessage.
 +
** Code of the form `x = condition ? a : b` with condition's value fixed for the block was optimized to `x = a`.
 +
** This may just be a relic of compiler upgrade mentioned above.
 +
* Minor changes to GetInfo:
 +
** Logic which previously disabled interrupt for profiling InfoTypes now instead disables dispatch (KScopedDisableDispatch instead of KScopedDisableInterrupt).
 +
** InfoType_ThreadTickCount has had its value changed from 0xF0000002 to 0x19. This is presumably to generate better asm for the switch statement.
 +
* SendAsyncRequestWithUserBuffer now accepts ResultThreadTerminating as a success result (it does not unlock memory when it is returned).
 +
* KClientSession::SendSyncRequest/SendAsyncRequest no longer hold the scheduler lock while calling KServerSession::OnRequest.
 +
** Lock is now acquired as needed by KServerSession::OnRequest itself.
 +
* KClientSession, KLightClientSession, KLightServerSession no longer inherit from KSlabAllocated/KAutoObjectWithList, and are now correspondingly smaller.
 +
* KResourceLimit::Allocate() now calls KAutoObject::Create().
 +
* KPrintf has been replaced with a function which takes a non-format string, and prints the string, then <current program ID formatted as %016lx>, then a newline.
 +
** "Break() called. %016lx\n" and "Exception occurred. %016lx\n" are now "Break() called. " and "Exception occurred. ", respectively.
 +
* StoreDataCacheSharedForInit has slightly different (but equivalent) iteration logic.
 +
    
===[[Bluetooth_Driver_services|bluetooth]]===
 
===[[Bluetooth_Driver_services|bluetooth]]===