14.0.0: Difference between revisions

Line 76:

=== BootImagePackage ===

All files in RomFs were updated.

====Secure Monitor====

* Compiler upgrade to latest llvm (now using same compiler revision as kernel).

** Secure Monitor is now compiled with -fomit-frame-pointer.

*** >:(

* GenerateSeTestVectorImpl now uses a helper to mix each key into the vector.

* ExceptionHandler is now linked in (@ .text + 0x3E04).

** Previously, this was garbage collected/only present in debug secure monitors.

** NOTE: This is unreachable, and stripped (as e.g. logging isn't emitted, likely because the macros are empty on release builds).

====Kernel====

* Kernel is now compiled with -O3 again instead of -Os

** >:(

* crt0 no longer supports booting in EL2.

** Infinite Loop/Panic is performed instead.

* Initialize0 changes:

** KernelStack setup now uses same helper to determine aslr as other random aligned regions.

** KernelTemp setup now uses same helper to determine aslr as other random aligned regions.

* Slab changes:

** When assigned extra resource, the slab heap is now 0x148000 larger instead of 0x68000 larger.

** Correspondingly, instead of increasing the thread resource limit by 160, the thread resource limit is now increased by 736.

*** This corresponds to changes in userland for pm management of resource limits.

*** Old Intended Resource Limits:

**** System (96 + 512) -> (256 + 512)

**** Applet 96 -> 96

**** Application 96 -> 96

*** New Intended Resource Limits:

**** System (96 + 512) -> 1024

**** Applet 96 -> 256

**** Application 96 -> 256

* SetupPoolPartitionMemoryRegions now panics if the end of the pool partition region is not coincidence with the end of dram.

* KThreadContext was completely revised.

** Most of KThreadContext is now stored inline in kernel stack.

*** Kernel stack layout is now u8 stack[0xDB0]; KThreadContext thread_context; KThreadStackParameters stack_parameters;

** KThreadContext now only stores the 8 callee-save FPU registers.

*** The remaining 24 caller-save FPU registers are stored inside KThread, where KThreadContext used to be.

*** NOTE that 32-bit fpu has 4 callee-save FPU registers and 12 caller-save registers, which use the start of the relevant 64-bit storages as usual.

** KThreadStackParameters was revised to facilitate this.

*** The pointer to KThreadContext previously stored in stack parameters now points to the external FPU register array.

*** The members at end of params are now: u16 disable_count; u8 current_svc_id; u8 unused_2c; u8 exception_flags; u8 is_pinned; u8 unused_2f;

**** The "exception_flags" field is a new set of bitflags (encoding old state was were previously separate bools + new state).

***** Bit 0x1 = is_calling_svc

***** Bit 0x2 = is_in_exception_handler

***** Bit 0x4 = is_fpu_state_restore_needed

***** Bit 0x8 = is_64_bit_fpu

***** Bit 0x10 = has_exception_svc_permissions

***** Bit 0x20 = is_in_cache_operation

***** Bit 0x40 = is_in_tlb_operation

** Exception exits now check is_fpu_state_restore_needed, and restore FPU registers only if needed (and clear is_fpu_state_restore_needed on restore).

*** is_fpu_state_restore_needed is set to true *only* on thread switch with FPU enabled.

**** Caller-save FPU registers are saved *only* if a thread is in an SVC and does not have exception svc permissions.

**** All other thread switches save only the 8 (or 4) callee-save FPU registers.

**** All thread switches now guarantee as post-condition that the fpu is disabled leaving the switch (it will be re-enabled on exception exit if needed).

*** On SVC exception return, all caller-save FPU registers are set to zero unless the thread has exception svc permissions.

** KThread::CloneFpuStatus now uses KScopedDisableInterrupt

* Various hw maintenance changes:

** KernelLdr no longer does cache maintenance by set/way when setting up initial identity mapping, no longer invalidates instruction cache/tlb, no longer does dsb after setting sctlr_el1.

** FlushEntireDataCacheLocal/Shared in init now perform dsb sy, FlushEntireDataCacheAndInvalidateTlbForInit no longer does after calling them.

** dsb sy/isb is now performed after setting sctlr_el1, when disabling mmu/icache.

** KInitialPageTable::Map no longer does dsb ish after all attribute writes.

*** Instead does it before writing table entries, and at the end of the function.

** KInitialPageTable::PhysicallyRandomize no longer does StoreEntireCacheForInit.

*** Now does dc cvac on randomized virtual address range, dsb ish, ic iallu, dsb ish, isb. (see weaker-barriers section of diff)

** KInitialPageTable::SwapBlocks now does dsb ish after memcpy to swap blocks.

** KInitialPageTable::Reprotect no longer does dsb ish before performing reprotection.

** KInitialProcessReader::Load no longer calls cpu::FlushEntireDataCache/cpu::InvalidateInstructionCache.

** Set/way cache operations now perform dsb sy before configuring csselr.

*** This affects InvalidateDataCacheForResumeEntry, FlushEntireDataCache, KCacheHelperInterruptHandler, and the initial cache maintenance when disabling the mmu.

** FlushEntireDataCache now does dsb sy after doing full set/way cache flush, instead of after each set/way op.

*** NOTE: This is still only a local flush without coherence guarantees, set/way aren't supposed to be used after multiple cores are online.

** KSystemControl::CpuSleepHandler no longer embeds unreachable cache maintenance assembly after CpuSuspend.

** Kernel now performs different hw maintenance if a thread is in a hw maintenance operation when interrupted:

*** If a thread is interrupted while performing cache maintenance in EL1 (tracked via new exception flags bit 0x20), KInterruptManager::OnHandleInterrupt performs dsb sy.

**** Set and cleared for scope of cpu::InvalidateDataCache instead of disabling core migration.

**** Set and cleared for scope of cpu::StoreDataCache instead of disabling core migration.

**** Set and cleared for scope of cpu::FlushDataCache instead of disabling core migration.

*** If a thread is interrupted while performing tlb maintenance in EL1 (tracked via new exception flags bit 0x40), KInterruptManager::OnHandleInterrupt performs dsb ish.

**** Set and cleared for scope of KPageTable::NoteUpdated

*** If a thread is interrupted while performing cache maintenance in EL0 (tracked via new bool @ TLS + 0x104), KInterruptManager::OnHandleInterrupt performs dsb sy.

**** This is equivalent to the EL1 cache maintenance tracking above, providing an opt-in way for userland to ensure its cache maintenance is coherent even when interrupted.

**** Note that official userland code now sets this bit before performing cache maintenance.

** Memory barriers were revised in many places -- barriers were weakened in many places, and some functions which previously lacked barriers had them added, including:

*** cpu::InvalidateEntireInstructionCache: dsb sy -> dsb ish

*** cpu::EnsureInstructionConsistency: dsb sy; isb; -> dsb ish; isb;

**** NOTE: Functions written in assembly still use the old pattern for ensuring instruction consistency.

*** KCacheInterruptHandler::RequestOperation: dsb sy -> dsb ish

*** KScheduler::EnableScheduling: dsb sy -> dsb ish

*** KScheduler::SwitchThread no longer does dsb sy before setting ttbr0/contextidr_el1.

*** KPageTable::NoteUpdated: dsb sy; if (m_kernel) { ... dsb sy; } else { ... dsb sy; isb; } -- dsb ishst; if (m_kernel) { ... dsb ish; } else { ... dsb ish; isb; }

**** KPageTable::NoteSingleKernelPageUpdated now similarly does dsb ishst for outer and dsb ish for inner barriers.

*** KPageTable::ClearPageTable: now does dsb ish after clearing page to zero via dc zva

*** KPageTable::MapContiguous: now does dsb ishst after merging pages.

*** KPageTable::MapPageGroup: now does dsb ishst after merging pages.

*** KPageTable::PteDataSynchronizationBarrier: now dmb ishst instead of dsb ish (probably KPageTable::PteDataMemoryBarrier, now?)

*** KPageTable::MapL2Blocks/MapL3Blocks: pattern for setting entry for new table went from Barrier(); WriteEntry(); Barrier(); -> Barrier(); WriteEntry();

**** This was PteDataSynchronizationBarrier(), and correspondingly asm is dsb ish; str; dsb ish; -> dmb ishst; str;

*** KSupervisorPageTable::SetTtbr0 no longer does dsb sy before setting ttbr0/contextidr_el1.

** UserspaceAccess::InvalidateInstructionCache was removed (previously unused).

* Various changes to KInterruptName/interrupt management:

** Enum values for IPIs were revised:

*** KInterruptName_ThreadTerminate 4 -> 0

*** KInterruptName_CacheOperation 5 -> 1

*** KInterruptName_Scheduler 6 -> 2

** New KInterruptName (KInterruptName_CoreBarrier) = 3

*** Interrupt handler for this is registered with KInterruptControllerPriority_Scheduler after ThreadTerminate handler is registered.

** Interrupt handler for the user cycle counter interrupt is no longer registered.

*** This is presumably now under the same ifdef that enables svc::InfoType_PerformanceCounter.

* KCapability now has a new member "physical_core_mask", which tracks what physical cores are allowable.

** KThread::FinishTermination now calls a new function (cpu::ForceSynchronizeAllCores) after waiting for the thread to not be current on any scheduler.

*** This function sends an IPI (KInterruptName_CoreBarrier) to all cores, and waits for all cores to acknowledge the interrupt.

* Changes to KMemoryManager allocation:

** KPageHeap now has an additional KPageHeapBitmapRng @ 0x328 to facilitate additional allocation randomization.

** KMemoryManager::AllocateAndOpenContinuous now uses a new KPageHeap method "AllocateRandomBlock"

*** KPhysicalAddress KPageHeap::AllocateRandomBlock(s32 index, size_t num_pages, size_t align_pages);

*** This method allocates `num_pages` pages (aligned to at least `align_pages`) at random.

**** First, the kernel chooses a random block index to allocate from.

***** This is done by increasing the block index until there are at least 4 possible random choices for the desired alignment, then selecting the block that corresponds to a random pick from those choices.

**** Next, the kernel allocates a random block from within that index.

**** Finally, the kernel selects a random (align_pages)-aligned offset within that block, frees the memory before/after the allocated chunk, and returns the memory.

** Allocation of KPageGroups still uses a `random` argument, however:

*** KPageHeap::PopBlock no longer takes a random argument.

*** KPageHeap::AllocateBlock now calls new new KPageHeap method "AllocateRandomBlock".

**** KPageHeap::AllocateRandomBlock(s32 index, size_t num_pages);

**** This is effectively the same logic as above, but with align_pages == # of pages for the argument block index.

* CreateProcess now calls a new function to validate the user-capabilities before creating the KProcess.

** This checks that the capabilities are user-readable and that the map region capabilities correspond to actually-present regions.

** This corresponds to changes in Loader allowing for map region capabilities (previously, these were only allowed via KIP, and Loader always rejected them).

* New InfoType 0x1A ("InfoType_IsSvcPermitted").

** Returns whether the current process can access a given SVC.

** Nintendo returns InvalidCombination when checking SVCs other than SynchronizePreemptionState.

*** Official userland code now aborts if the process does not have permission to use SynchronizePreemptionState before incrementing ThreadLocalRegion->disable_count for the first time.

==See Also==

@@ Line 76: / Line 76: @@
 === BootImagePackage ===
 All files in RomFs were updated.
+====Secure Monitor====
+* Compiler upgrade to latest llvm (now using same compiler revision as kernel).
+** Secure Monitor is now compiled with -fomit-frame-pointer.
+*** >:(
+* GenerateSeTestVectorImpl now uses a helper to mix each key into the vector.
+* ExceptionHandler is now linked in (@ .text + 0x3E04).
+** Previously, this was garbage collected/only present in debug secure monitors.
+** NOTE: This is unreachable, and stripped (as e.g. logging isn't emitted, likely because the macros are empty on release builds).
+====Kernel====
+* Kernel is now compiled with -O3 again instead of -Os
+** >:(
+* crt0 no longer supports booting in EL2.
+** Infinite Loop/Panic is performed instead.
+* Initialize0 changes:
+** KernelStack setup now uses same helper to determine aslr as other random aligned regions.
+** KernelTemp setup now uses same helper to determine aslr as other random aligned regions.
+* Slab changes:
+** When assigned extra resource, the slab heap is now 0x148000 larger instead of 0x68000 larger.
+** Correspondingly, instead of increasing the thread resource limit by 160, the thread resource limit is now increased by 736.
+*** This corresponds to changes in userland for pm management of resource limits.
+*** Old Intended Resource Limits:
+**** System (96 + 512) -> (256 + 512)
+**** Applet 96 -> 96
+**** Application 96 -> 96
+*** New Intended Resource Limits:
+**** System (96 + 512) -> 1024
+**** Applet 96 -> 256
+**** Application 96 -> 256
+* SetupPoolPartitionMemoryRegions now panics if the end of the pool partition region is not coincidence with the end of dram.
+* KThreadContext was completely revised.
+** Most of KThreadContext is now stored inline in kernel stack.
+*** Kernel stack layout is now u8 stack[0xDB0]; KThreadContext thread_context; KThreadStackParameters stack_parameters;
+** KThreadContext now only stores the 8 callee-save FPU registers.
+*** The remaining 24 caller-save FPU registers are stored inside KThread, where KThreadContext used to be.
+*** NOTE that 32-bit fpu has 4 callee-save FPU registers and 12 caller-save registers, which use the start of the relevant 64-bit storages as usual.
+** KThreadStackParameters was revised to facilitate this.
+*** The pointer to KThreadContext previously stored in stack parameters now points to the external FPU register array.
+*** The members at end of params are now: u16 disable_count; u8 current_svc_id; u8 unused_2c; u8 exception_flags; u8 is_pinned; u8 unused_2f;
+**** The "exception_flags" field is a new set of bitflags (encoding old state was were previously separate bools + new state).
+***** Bit 0x1  = is_calling_svc
+***** Bit 0x2  = is_in_exception_handler
+***** Bit 0x4  = is_fpu_state_restore_needed
+***** Bit 0x8  = is_64_bit_fpu
+***** Bit 0x10 = has_exception_svc_permissions
+***** Bit 0x20 = is_in_cache_operation
+***** Bit 0x40 = is_in_tlb_operation
+** Exception exits now check is_fpu_state_restore_needed, and restore FPU registers only if needed (and clear is_fpu_state_restore_needed on restore).
+*** is_fpu_state_restore_needed is set to true *only* on thread switch with FPU enabled.
+**** Caller-save FPU registers are saved *only* if a thread is in an SVC and does not have exception svc permissions.
+**** All other thread switches save only the 8 (or 4) callee-save FPU registers.
+**** All thread switches now guarantee as post-condition that the fpu is disabled leaving the switch (it will be re-enabled on exception exit if needed).
+*** On SVC exception return, all caller-save FPU registers are set to zero unless the thread has exception svc permissions.
+** KThread::CloneFpuStatus now uses KScopedDisableInterrupt
+* Various hw maintenance changes:
+** KernelLdr no longer does cache maintenance by set/way when setting up initial identity mapping, no longer invalidates instruction cache/tlb, no longer does dsb after setting sctlr_el1.
+** FlushEntireDataCacheLocal/Shared in init now perform dsb sy, FlushEntireDataCacheAndInvalidateTlbForInit no longer does after calling them.
+** dsb sy/isb is now performed after setting sctlr_el1, when disabling mmu/icache.
+** KInitialPageTable::Map no longer does dsb ish after all attribute writes.
+*** Instead does it before writing table entries, and at the end of the function.
+** KInitialPageTable::PhysicallyRandomize no longer does StoreEntireCacheForInit.
+*** Now does dc cvac on randomized virtual address range, dsb ish, ic iallu, dsb ish, isb. (see weaker-barriers section of diff)
+** KInitialPageTable::SwapBlocks now does dsb ish after memcpy to swap blocks.
+** KInitialPageTable::Reprotect no longer does dsb ish before performing reprotection.
+** KInitialProcessReader::Load no longer calls cpu::FlushEntireDataCache/cpu::InvalidateInstructionCache.
+** Set/way cache operations now perform dsb sy before configuring csselr.
+*** This affects InvalidateDataCacheForResumeEntry, FlushEntireDataCache, KCacheHelperInterruptHandler, and the initial cache maintenance when disabling the mmu.
+** FlushEntireDataCache now does dsb sy after doing full set/way cache flush, instead of after each set/way op.
+*** NOTE: This is still only a local flush without coherence guarantees, set/way aren't supposed to be used after multiple cores are online.
+** KSystemControl::CpuSleepHandler no longer embeds unreachable cache maintenance assembly after CpuSuspend.
+** Kernel now performs different hw maintenance if a thread is in a hw maintenance operation when interrupted:
+*** If a thread is interrupted while performing cache maintenance in EL1 (tracked via new exception flags bit 0x20), KInterruptManager::OnHandleInterrupt performs dsb sy.
+**** Set and cleared for scope of cpu::InvalidateDataCache instead of disabling core migration.
+**** Set and cleared for scope of cpu::StoreDataCache instead of disabling core migration.
+**** Set and cleared for scope of cpu::FlushDataCache instead of disabling core migration.
+*** If a thread is interrupted while performing tlb maintenance in EL1 (tracked via new exception flags bit 0x40), KInterruptManager::OnHandleInterrupt performs dsb ish.
+**** Set and cleared for scope of KPageTable::NoteUpdated
+*** If a thread is interrupted while performing cache maintenance in EL0 (tracked via new bool @ TLS + 0x104), KInterruptManager::OnHandleInterrupt performs dsb sy.
+**** This is equivalent to the EL1 cache maintenance tracking above, providing an opt-in way for userland to ensure its cache maintenance is coherent even when interrupted.
+**** Note that official userland code now sets this bit before performing cache maintenance.
+** Memory barriers were revised in many places -- barriers were weakened in many places, and some functions which previously lacked barriers had them added, including:
+*** cpu::InvalidateEntireInstructionCache: dsb sy -> dsb ish
+*** cpu::EnsureInstructionConsistency: dsb sy; isb; -> dsb ish; isb;
+**** NOTE: Functions written in assembly still use the old pattern for ensuring instruction consistency.
+*** KCacheInterruptHandler::RequestOperation: dsb sy -> dsb ish
+*** KScheduler::EnableScheduling: dsb sy -> dsb ish
+*** KScheduler::SwitchThread no longer does dsb sy before setting ttbr0/contextidr_el1.
+*** KPageTable::NoteUpdated: dsb sy; if (m_kernel) { ... dsb sy; } else { ... dsb sy; isb; } -- dsb ishst; if (m_kernel) { ... dsb ish; } else { ... dsb ish; isb; }
+**** KPageTable::NoteSingleKernelPageUpdated now similarly does dsb ishst for outer and dsb ish for inner barriers.
+*** KPageTable::ClearPageTable: now does dsb ish after clearing page to zero via dc zva
+*** KPageTable::MapContiguous: now does dsb ishst after merging pages.
+*** KPageTable::MapPageGroup: now does dsb ishst after merging pages.
+*** KPageTable::PteDataSynchronizationBarrier: now dmb ishst instead of dsb ish (probably KPageTable::PteDataMemoryBarrier, now?)
+*** KPageTable::MapL2Blocks/MapL3Blocks: pattern for setting entry for new table went from Barrier(); WriteEntry(); Barrier(); -> Barrier(); WriteEntry();
+**** This was PteDataSynchronizationBarrier(), and correspondingly asm is dsb ish; str; dsb ish; -> dmb ishst; str;
+*** KSupervisorPageTable::SetTtbr0 no longer does dsb sy before setting ttbr0/contextidr_el1.
+** UserspaceAccess::InvalidateInstructionCache was removed (previously unused).
+* Various changes to KInterruptName/interrupt management:
+** Enum values for IPIs were revised:
+*** KInterruptName_ThreadTerminate    4 -> 0
+*** KInterruptName_CacheOperation     5 -> 1
+*** KInterruptName_Scheduler          6 -> 2
+** New KInterruptName (KInterruptName_CoreBarrier) = 3
+*** Interrupt handler for this is registered with KInterruptControllerPriority_Scheduler after ThreadTerminate handler is registered.
+** Interrupt handler for the user cycle counter interrupt is no longer registered.
+*** This is presumably now under the same ifdef that enables svc::InfoType_PerformanceCounter.
+* KCapability now has a new member "physical_core_mask", which tracks what physical cores are allowable.
+** KThread::FinishTermination now calls a new function (cpu::ForceSynchronizeAllCores) after waiting for the thread to not be current on any scheduler.
+*** This function sends an IPI (KInterruptName_CoreBarrier) to all cores, and waits for all cores to acknowledge the interrupt.
+* Changes to KMemoryManager allocation:
+** KPageHeap now has an additional KPageHeapBitmapRng @ 0x328 to facilitate additional allocation randomization.
+** KMemoryManager::AllocateAndOpenContinuous now uses a new KPageHeap method "AllocateRandomBlock"
+*** KPhysicalAddress KPageHeap::AllocateRandomBlock(s32 index, size_t num_pages, size_t align_pages);
+*** This method allocates `num_pages` pages (aligned to at least `align_pages`) at random.
+**** First, the kernel chooses a random block index to allocate from.
+***** This is done by increasing the block index until there are at least 4 possible random choices for the desired alignment, then selecting the block that corresponds to a random pick from those choices.
+**** Next, the kernel allocates a random block from within that index.
+**** Finally, the kernel selects a random (align_pages)-aligned offset within that block, frees the memory before/after the allocated chunk, and returns the memory.
+** Allocation of KPageGroups still uses a `random` argument, however:
+*** KPageHeap::PopBlock no longer takes a random argument.
+*** KPageHeap::AllocateBlock now calls new new KPageHeap method "AllocateRandomBlock".
+**** KPageHeap::AllocateRandomBlock(s32 index, size_t num_pages);
+**** This is effectively the same logic as above, but with align_pages == # of pages for the argument block index.
+* CreateProcess now calls a new function to validate the user-capabilities before creating the KProcess.
+** This checks that the capabilities are user-readable and that the map region capabilities correspond to actually-present regions.
+** This corresponds to changes in Loader allowing for map region capabilities (previously, these were only allowed via KIP, and Loader always rejected them).
+* New InfoType 0x1A ("InfoType_IsSvcPermitted").
+** Returns whether the current process can access a given SVC.
+** Nintendo returns InvalidCombination when checking SVCs other than SynchronizePreemptionState.
+*** Official userland code now aborts if the process does not have permission to use SynchronizePreemptionState before incrementing ThreadLocalRegion->disable_count for the first time.
 ==See Also==