Compare commits
14 Commits
ceedd21aee
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
1fc622c5da | ||
|
|
e754bac5e8 | ||
| 459c0caca7 | |||
| 6fd5085ed1 | |||
| c4575242e0 | |||
| 957ed987fe | |||
| 0c3d9b8370 | |||
|
|
1bfde637d0 | ||
|
|
c092a81331 | ||
|
|
1b38c7f445 | ||
| f5dd56a379 | |||
|
|
e3ac9612c9 | ||
|
|
b59779ca4a | ||
|
|
3f87a93d52 |
808
LICENSE
808
LICENSE
@@ -1,202 +1,674 @@
|
||||
GNU GENERAL PUBLIC LICENSE
|
||||
Version 3, 29 June 2007
|
||||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||
Everyone is permitted to copy and distribute verbatim copies
|
||||
of this license document, but changing it is not allowed.
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
Preamble
|
||||
|
||||
1. Definitions.
|
||||
The GNU General Public License is a free, copyleft license for
|
||||
software and other kinds of works.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
The licenses for most software and other practical works are designed
|
||||
to take away your freedom to share and change the works. By contrast,
|
||||
the GNU General Public License is intended to guarantee your freedom to
|
||||
share and change all versions of a program--to make sure it remains free
|
||||
software for all its users. We, the Free Software Foundation, use the
|
||||
GNU General Public License for most of our software; it applies also to
|
||||
any other work released this way by its authors. You can apply it to
|
||||
your programs, too.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
When we speak of free software, we are referring to freedom, not
|
||||
price. Our General Public Licenses are designed to make sure that you
|
||||
have the freedom to distribute copies of free software (and charge for
|
||||
them if you wish), that you receive source code or can get it if you
|
||||
want it, that you can change the software or use pieces of it in new
|
||||
free programs, and that you know you can do these things.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
To protect your rights, we need to prevent others from denying you
|
||||
these rights or asking you to surrender the rights. Therefore, you have
|
||||
certain responsibilities if you distribute copies of the software, or if
|
||||
you modify it: responsibilities to respect the freedom of others.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
For example, if you distribute copies of such a program, whether
|
||||
gratis or for a fee, you must pass on to the recipients the same
|
||||
freedoms that you received. You must make sure that they, too, receive
|
||||
or can get the source code. And you must show them these terms so they
|
||||
know their rights.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
Developers that use the GNU GPL protect your rights with two steps:
|
||||
(1) assert copyright on the software, and (2) offer you this License
|
||||
giving you legal permission to copy, distribute and/or modify it.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
For the developers' and authors' protection, the GPL clearly explains
|
||||
that there is no warranty for this free software. For both users' and
|
||||
authors' sake, the GPL requires that modified versions be marked as
|
||||
changed, so that their problems will not be attributed erroneously to
|
||||
authors of previous versions.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
Some devices are designed to deny users access to install or run
|
||||
modified versions of the software inside them, although the manufacturer
|
||||
can do so. This is fundamentally incompatible with the aim of
|
||||
protecting users' freedom to change the software. The systematic
|
||||
pattern of such abuse occurs in the area of products for individuals to
|
||||
use, which is precisely where it is most unacceptable. Therefore, we
|
||||
have designed this version of the GPL to prohibit the practice for those
|
||||
products. If such problems arise substantially in other domains, we
|
||||
stand ready to extend this provision to those domains in future versions
|
||||
of the GPL, as needed to protect the freedom of users.
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
Finally, every program is threatened constantly by software patents.
|
||||
States should not allow patents to restrict development and use of
|
||||
software on general-purpose computers, but in those that do, we wish to
|
||||
avoid the special danger that patents applied to a free program could
|
||||
make it effectively proprietary. To prevent this, the GPL assures that
|
||||
patents cannot be used to render the program non-free.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
The precise terms and conditions for copying, distribution and
|
||||
modification follow.
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
TERMS AND CONDITIONS
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
0. Definitions.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
"This License" refers to version 3 of the GNU General Public License.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
"Copyright" also means copyright-like laws that apply to other kinds of
|
||||
works, such as semiconductor masks.
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
"The Program" refers to any copyrightable work licensed under this
|
||||
License. Each licensee is addressed as "you". "Licensees" and
|
||||
"recipients" may be individuals or organizations.
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
To "modify" a work means to copy from or adapt all or part of the work
|
||||
in a fashion requiring copyright permission, other than the making of an
|
||||
exact copy. The resulting work is called a "modified version" of the
|
||||
earlier work or a work "based on" the earlier work.
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
A "covered work" means either the unmodified Program or a work based
|
||||
on the Program.
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
To "propagate" a work means to do anything with it that, without
|
||||
permission, would make you directly or secondarily liable for
|
||||
infringement under applicable copyright law, except executing it on a
|
||||
computer or modifying a private copy. Propagation includes copying,
|
||||
distribution (with or without modification), making available to the
|
||||
public, and in some countries other activities as well.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
To "convey" a work means any kind of propagation that enables other
|
||||
parties to make or receive copies. Mere interaction with a user through
|
||||
a computer network, with no transfer of a copy, is not conveying.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
An interactive user interface displays "Appropriate Legal Notices"
|
||||
to the extent that it includes a convenient and prominently visible
|
||||
feature that (1) displays an appropriate copyright notice, and (2)
|
||||
tells the user that there is no warranty for the work (except to the
|
||||
extent that warranties are provided), that licensees may convey the
|
||||
work under this License, and how to view a copy of this License. If
|
||||
the interface presents a list of user commands or options, such as a
|
||||
menu, a prominent item in the list meets this criterion.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
1. Source Code.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
The "source code" for a work means the preferred form of the work
|
||||
for making modifications to it. "Object code" means any non-source
|
||||
form of a work.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
A "Standard Interface" means an interface that either is an official
|
||||
standard defined by a recognized standards body, or, in the case of
|
||||
interfaces specified for a particular programming language, one that
|
||||
is widely used among developers working in that language.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
The "System Libraries" of an executable work include anything, other
|
||||
than the work as a whole, that (a) is included in the normal form of
|
||||
packaging a Major Component, but which is not part of that Major
|
||||
Component, and (b) serves only to enable use of the work with that
|
||||
Major Component, or to implement a Standard Interface for which an
|
||||
implementation is available to the public in source code form. A
|
||||
"Major Component", in this context, means a major essential component
|
||||
(kernel, window system, and so on) of the specific operating system
|
||||
(if any) on which the executable work runs, or a compiler used to
|
||||
produce the work, or an object code interpreter used to run it.
|
||||
|
||||
The "Corresponding Source" for a work in object code form means all
|
||||
the source code needed to generate, install, and (for an executable
|
||||
work) run the object code and to modify the work, including scripts to
|
||||
control those activities. However, it does not include the work's
|
||||
System Libraries, or general-purpose tools or generally available free
|
||||
programs which are used unmodified in performing those activities but
|
||||
which are not part of the work. For example, Corresponding Source
|
||||
includes interface definition files associated with source files for
|
||||
the work, and the source code for shared libraries and dynamically
|
||||
linked subprograms that the work is specifically designed to require,
|
||||
such as by intimate data communication or control flow between those
|
||||
subprograms and other parts of the work.
|
||||
|
||||
The Corresponding Source need not include anything that users
|
||||
can regenerate automatically from other parts of the Corresponding
|
||||
Source.
|
||||
|
||||
The Corresponding Source for a work in source code form is that
|
||||
same work.
|
||||
|
||||
2. Basic Permissions.
|
||||
|
||||
All rights granted under this License are granted for the term of
|
||||
copyright on the Program, and are irrevocable provided the stated
|
||||
conditions are met. This License explicitly affirms your unlimited
|
||||
permission to run the unmodified Program. The output from running a
|
||||
covered work is covered by this License only if the output, given its
|
||||
content, constitutes a covered work. This License acknowledges your
|
||||
rights of fair use or other equivalent, as provided by copyright law.
|
||||
|
||||
You may make, run and propagate covered works that you do not
|
||||
convey, without conditions so long as your license otherwise remains
|
||||
in force. You may convey covered works to others for the sole purpose
|
||||
of having them make modifications exclusively for you, or provide you
|
||||
with facilities for running those works, provided that you comply with
|
||||
the terms of this License in conveying all material for which you do
|
||||
not control copyright. Those thus making or running the covered works
|
||||
for you must do so exclusively on your behalf, under your direction
|
||||
and control, on terms that prohibit them from making any copies of
|
||||
your copyrighted material outside their relationship with you.
|
||||
|
||||
Conveying under any other circumstances is permitted solely under
|
||||
the conditions stated below. Sublicensing is not allowed; section 10
|
||||
makes it unnecessary.
|
||||
|
||||
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||
|
||||
No covered work shall be deemed part of an effective technological
|
||||
measure under any applicable law fulfilling obligations under article
|
||||
11 of the WIPO copyright treaty adopted on 20 December 1996, or
|
||||
similar laws prohibiting or restricting circumvention of such
|
||||
measures.
|
||||
|
||||
When you convey a covered work, you waive any legal power to forbid
|
||||
circumvention of technological measures to the extent such circumvention
|
||||
is effected by exercising rights under this License with respect to
|
||||
the covered work, and you disclaim any intention to limit operation or
|
||||
modification of the work as a means of enforcing, against the work's
|
||||
users, your or third parties' legal rights to forbid circumvention of
|
||||
technological measures.
|
||||
|
||||
4. Conveying Verbatim Copies.
|
||||
|
||||
You may convey verbatim copies of the Program's source code as you
|
||||
receive it, in any medium, provided that you conspicuously and
|
||||
appropriately publish on each copy an appropriate copyright notice;
|
||||
keep intact all notices stating that this License and any
|
||||
non-permissive terms added in accord with section 7 apply to the code;
|
||||
keep intact all notices of the absence of any warranty; and give all
|
||||
recipients a copy of this License along with the Program.
|
||||
|
||||
You may charge any price or no price for each copy that you convey,
|
||||
and you may offer support or warranty protection for a fee.
|
||||
|
||||
5. Conveying Modified Source Versions.
|
||||
|
||||
You may convey a work based on the Program, or the modifications to
|
||||
produce it from the Program, in the form of source code under the
|
||||
terms of section 4, provided that you also meet all of these conditions:
|
||||
|
||||
a) The work must carry prominent notices stating that you modified
|
||||
it, and giving a relevant date.
|
||||
|
||||
b) The work must carry prominent notices stating that it is
|
||||
released under this License and any conditions added under section
|
||||
7. This requirement modifies the requirement in section 4 to
|
||||
"keep intact all notices".
|
||||
|
||||
c) You must license the entire work, as a whole, under this
|
||||
License to anyone who comes into possession of a copy. This
|
||||
License will therefore apply, along with any applicable section 7
|
||||
additional terms, to the whole of the work, and all its parts,
|
||||
regardless of how they are packaged. This License gives no
|
||||
permission to license the work in any other way, but it does not
|
||||
invalidate such permission if you have separately received it.
|
||||
|
||||
d) If the work has interactive user interfaces, each must display
|
||||
Appropriate Legal Notices; however, if the Program has interactive
|
||||
interfaces that do not display Appropriate Legal Notices, your
|
||||
work need not make them do so.
|
||||
|
||||
A compilation of a covered work with other separate and independent
|
||||
works, which are not by their nature extensions of the covered work,
|
||||
and which are not combined with it such as to form a larger program,
|
||||
in or on a volume of a storage or distribution medium, is called an
|
||||
"aggregate" if the compilation and its resulting copyright are not
|
||||
used to limit the access or legal rights of the compilation's users
|
||||
beyond what the individual works permit. Inclusion of a covered work
|
||||
in an aggregate does not cause this License to apply to the other
|
||||
parts of the aggregate.
|
||||
|
||||
6. Conveying Non-Source Forms.
|
||||
|
||||
You may convey a covered work in object code form under the terms
|
||||
of sections 4 and 5, provided that you also convey the
|
||||
machine-readable Corresponding Source under the terms of this License,
|
||||
in one of these ways:
|
||||
|
||||
a) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by the
|
||||
Corresponding Source fixed on a durable physical medium
|
||||
customarily used for software interchange.
|
||||
|
||||
b) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by a
|
||||
written offer, valid for at least three years and valid for as
|
||||
long as you offer spare parts or customer support for that product
|
||||
model, to give anyone who possesses the object code either (1) a
|
||||
copy of the Corresponding Source for all the software in the
|
||||
product that is covered by this License, on a durable physical
|
||||
medium customarily used for software interchange, for a price no
|
||||
more than your reasonable cost of physically performing this
|
||||
conveying of source, or (2) access to copy the
|
||||
Corresponding Source from a network server at no charge.
|
||||
|
||||
c) Convey individual copies of the object code with a copy of the
|
||||
written offer to provide the Corresponding Source. This
|
||||
alternative is allowed only occasionally and noncommercially, and
|
||||
only if you received the object code with such an offer, in accord
|
||||
with subsection 6b.
|
||||
|
||||
d) Convey the object code by offering access from a designated
|
||||
place (gratis or for a charge), and offer equivalent access to the
|
||||
Corresponding Source in the same way through the same place at no
|
||||
further charge. You need not require recipients to copy the
|
||||
Corresponding Source along with the object code. If the place to
|
||||
copy the object code is a network server, the Corresponding Source
|
||||
may be on a different server (operated by you or a third party)
|
||||
that supports equivalent copying facilities, provided you maintain
|
||||
clear directions next to the object code saying where to find the
|
||||
Corresponding Source. Regardless of what server hosts the
|
||||
Corresponding Source, you remain obligated to ensure that it is
|
||||
available for as long as needed to satisfy these requirements.
|
||||
|
||||
e) Convey the object code using peer-to-peer transmission, provided
|
||||
you inform other peers where the object code and Corresponding
|
||||
Source of the work are being offered to the general public at no
|
||||
charge under subsection 6d.
|
||||
|
||||
A separable portion of the object code, whose source code is excluded
|
||||
from the Corresponding Source as a System Library, need not be
|
||||
included in conveying the object code work.
|
||||
|
||||
A "User Product" is either (1) a "consumer product", which means any
|
||||
tangible personal property which is normally used for personal, family,
|
||||
or household purposes, or (2) anything designed or sold for incorporation
|
||||
into a dwelling. In determining whether a product is a consumer product,
|
||||
doubtful cases shall be resolved in favor of coverage. For a particular
|
||||
product received by a particular user, "normally used" refers to a
|
||||
typical or common use of that class of product, regardless of the status
|
||||
of the particular user or of the way in which the particular user
|
||||
actually uses, or expects or is expected to use, the product. A product
|
||||
is a consumer product regardless of whether the product has substantial
|
||||
commercial, industrial or non-consumer uses, unless such uses represent
|
||||
the only significant mode of use of the product.
|
||||
|
||||
"Installation Information" for a User Product means any methods,
|
||||
procedures, authorization keys, or other information required to install
|
||||
and execute modified versions of a covered work in that User Product from
|
||||
a modified version of its Corresponding Source. The information must
|
||||
suffice to ensure that the continued functioning of the modified object
|
||||
code is in no case prevented or interfered with solely because
|
||||
modification has been made.
|
||||
|
||||
If you convey an object code work under this section in, or with, or
|
||||
specifically for use in, a User Product, and the conveying occurs as
|
||||
part of a transaction in which the right of possession and use of the
|
||||
User Product is transferred to the recipient in perpetuity or for a
|
||||
fixed term (regardless of how the transaction is characterized), the
|
||||
Corresponding Source conveyed under this section must be accompanied
|
||||
by the Installation Information. But this requirement does not apply
|
||||
if neither you nor any third party retains the ability to install
|
||||
modified object code on the User Product (for example, the work has
|
||||
been installed in ROM).
|
||||
|
||||
The requirement to provide Installation Information does not include a
|
||||
requirement to continue to provide support service, warranty, or updates
|
||||
for a work that has been modified or installed by the recipient, or for
|
||||
the User Product in which it has been modified or installed. Access to a
|
||||
network may be denied when the modification itself materially and
|
||||
adversely affects the operation of the network or violates the rules and
|
||||
protocols for communication across the network.
|
||||
|
||||
Corresponding Source conveyed, and Installation Information provided,
|
||||
in accord with this section must be in a format that is publicly
|
||||
documented (and with an implementation available to the public in
|
||||
source code form), and must require no special password or key for
|
||||
unpacking, reading or copying.
|
||||
|
||||
7. Additional Terms.
|
||||
|
||||
"Additional permissions" are terms that supplement the terms of this
|
||||
License by making exceptions from one or more of its conditions.
|
||||
Additional permissions that are applicable to the entire Program shall
|
||||
be treated as though they were included in this License, to the extent
|
||||
that they are valid under applicable law. If additional permissions
|
||||
apply only to part of the Program, that part may be used separately
|
||||
under those permissions, but the entire Program remains governed by
|
||||
this License without regard to the additional permissions.
|
||||
|
||||
When you convey a copy of a covered work, you may at your option
|
||||
remove any additional permissions from that copy, or from any part of
|
||||
it. (Additional permissions may be written to require their own
|
||||
removal in certain cases when you modify the work.) You may place
|
||||
additional permissions on material, added by you to a covered work,
|
||||
for which you have or can give appropriate copyright permission.
|
||||
|
||||
Notwithstanding any other provision of this License, for material you
|
||||
add to a covered work, you may (if authorized by the copyright holders of
|
||||
that material) supplement the terms of this License with terms:
|
||||
|
||||
a) Disclaiming warranty or limiting liability differently from the
|
||||
terms of sections 15 and 16 of this License; or
|
||||
|
||||
b) Requiring preservation of specified reasonable legal notices or
|
||||
author attributions in that material or in the Appropriate Legal
|
||||
Notices displayed by works containing it; or
|
||||
|
||||
c) Prohibiting misrepresentation of the origin of that material, or
|
||||
requiring that modified versions of such material be marked in
|
||||
reasonable ways as different from the original version; or
|
||||
|
||||
d) Limiting the use for publicity purposes of names of licensors or
|
||||
authors of the material; or
|
||||
|
||||
e) Declining to grant rights under trademark law for use of some
|
||||
trade names, trademarks, or service marks; or
|
||||
|
||||
f) Requiring indemnification of licensors and authors of that
|
||||
material by anyone who conveys the material (or modified versions of
|
||||
it) with contractual assumptions of liability to the recipient, for
|
||||
any liability that these contractual assumptions directly impose on
|
||||
those licensors and authors.
|
||||
|
||||
All other non-permissive additional terms are considered "further
|
||||
restrictions" within the meaning of section 10. If the Program as you
|
||||
received it, or any part of it, contains a notice stating that it is
|
||||
governed by this License along with a term that is a further
|
||||
restriction, you may remove that term. If a license document contains
|
||||
a further restriction but permits relicensing or conveying under this
|
||||
License, you may add to a covered work material governed by the terms
|
||||
of that license document, provided that the further restriction does
|
||||
not survive such relicensing or conveying.
|
||||
|
||||
If you add terms to a covered work in accord with this section, you
|
||||
must place, in the relevant source files, a statement of the
|
||||
additional terms that apply to those files, or a notice indicating
|
||||
where to find the applicable terms.
|
||||
|
||||
Additional terms, permissive or non-permissive, may be stated in the
|
||||
form of a separately written license, or stated as exceptions;
|
||||
the above requirements apply either way.
|
||||
|
||||
8. Termination.
|
||||
|
||||
You may not propagate or modify a covered work except as expressly
|
||||
provided under this License. Any attempt otherwise to propagate or
|
||||
modify it is void, and will automatically terminate your rights under
|
||||
this License (including any patent licenses granted under the third
|
||||
paragraph of section 11).
|
||||
|
||||
However, if you cease all violation of this License, then your
|
||||
license from a particular copyright holder is reinstated (a)
|
||||
provisionally, unless and until the copyright holder explicitly and
|
||||
finally terminates your license, and (b) permanently, if the copyright
|
||||
holder fails to notify you of the violation by some reasonable means
|
||||
prior to 60 days after the cessation.
|
||||
|
||||
Moreover, your license from a particular copyright holder is
|
||||
reinstated permanently if the copyright holder notifies you of the
|
||||
violation by some reasonable means, this is the first time you have
|
||||
received notice of violation of this License (for any work) from that
|
||||
copyright holder, and you cure the violation prior to 30 days after
|
||||
your receipt of the notice.
|
||||
|
||||
Termination of your rights under this section does not terminate the
|
||||
licenses of parties who have received copies or rights from you under
|
||||
this License. If your rights have been terminated and not permanently
|
||||
reinstated, you do not qualify to receive new licenses for the same
|
||||
material under section 10.
|
||||
|
||||
9. Acceptance Not Required for Having Copies.
|
||||
|
||||
You are not required to accept this License in order to receive or
|
||||
run a copy of the Program. Ancillary propagation of a covered work
|
||||
occurring solely as a consequence of using peer-to-peer transmission
|
||||
to receive a copy likewise does not require acceptance. However,
|
||||
nothing other than this License grants you permission to propagate or
|
||||
modify any covered work. These actions infringe copyright if you do
|
||||
not accept this License. Therefore, by modifying or propagating a
|
||||
covered work, you indicate your acceptance of this License to do so.
|
||||
|
||||
10. Automatic Licensing of Downstream Recipients.
|
||||
|
||||
Each time you convey a covered work, the recipient automatically
|
||||
receives a license from the original licensors, to run, modify and
|
||||
propagate that work, subject to this License. You are not responsible
|
||||
for enforcing compliance by third parties with this License.
|
||||
|
||||
An "entity transaction" is a transaction transferring control of an
|
||||
organization, or substantially all assets of one, or subdividing an
|
||||
organization, or merging organizations. If propagation of a covered
|
||||
work results from an entity transaction, each party to that
|
||||
transaction who receives a copy of the work also receives whatever
|
||||
licenses to the work the party's predecessor in interest had or could
|
||||
give under the previous paragraph, plus a right to possession of the
|
||||
Corresponding Source of the work from the predecessor in interest, if
|
||||
the predecessor has it or can get it with reasonable efforts.
|
||||
|
||||
You may not impose any further restrictions on the exercise of the
|
||||
rights granted or affirmed under this License. For example, you may
|
||||
not impose a license fee, royalty, or other charge for exercise of
|
||||
rights granted under this License, and you may not initiate litigation
|
||||
(including a cross-claim or counterclaim in a lawsuit) alleging that
|
||||
any patent claim is infringed by making, using, selling, offering for
|
||||
sale, or importing the Program or any portion of it.
|
||||
|
||||
11. Patents.
|
||||
|
||||
A "contributor" is a copyright holder who authorizes use under this
|
||||
License of the Program or a work on which the Program is based. The
|
||||
work thus licensed is called the contributor's "contributor version".
|
||||
|
||||
A contributor's "essential patent claims" are all patent claims
|
||||
owned or controlled by the contributor, whether already acquired or
|
||||
hereafter acquired, that would be infringed by some manner, permitted
|
||||
by this License, of making, using, or selling its contributor version,
|
||||
but do not include claims that would be infringed only as a
|
||||
consequence of further modification of the contributor version. For
|
||||
purposes of this definition, "control" includes the right to grant
|
||||
patent sublicenses in a manner consistent with the requirements of
|
||||
this License.
|
||||
|
||||
Each contributor grants you a non-exclusive, worldwide, royalty-free
|
||||
patent license under the contributor's essential patent claims, to
|
||||
make, use, sell, offer for sale, import and otherwise run, modify and
|
||||
propagate the contents of its contributor version.
|
||||
|
||||
In the following three paragraphs, a "patent license" is any express
|
||||
agreement or commitment, however denominated, not to enforce a patent
|
||||
(such as an express permission to practice a patent or covenant not to
|
||||
sue for patent infringement). To "grant" such a patent license to a
|
||||
party means to make such an agreement or commitment not to enforce a
|
||||
patent against the party.
|
||||
|
||||
If you convey a covered work, knowingly relying on a patent license,
|
||||
and the Corresponding Source of the work is not available for anyone
|
||||
to copy, free of charge and under the terms of this License, through a
|
||||
publicly available network server or other readily accessible means,
|
||||
then you must either (1) cause the Corresponding Source to be so
|
||||
available, or (2) arrange to deprive yourself of the benefit of the
|
||||
patent license for this particular work, or (3) arrange, in a manner
|
||||
consistent with the requirements of this License, to extend the patent
|
||||
license to downstream recipients. "Knowingly relying" means you have
|
||||
actual knowledge that, but for the patent license, your conveying the
|
||||
covered work in a country, or your recipient's use of the covered work
|
||||
in a country, would infringe one or more identifiable patents in that
|
||||
country that you have reason to believe are valid.
|
||||
|
||||
If, pursuant to or in connection with a single transaction or
|
||||
arrangement, you convey, or propagate by procuring conveyance of, a
|
||||
covered work, and grant a patent license to some of the parties
|
||||
receiving the covered work authorizing them to use, propagate, modify
|
||||
or convey a specific copy of the covered work, then the patent license
|
||||
you grant is automatically extended to all recipients of the covered
|
||||
work and works based on it.
|
||||
|
||||
A patent license is "discriminatory" if it does not include within
|
||||
the scope of its coverage, prohibits the exercise of, or is
|
||||
conditioned on the non-exercise of one or more of the rights that are
|
||||
specifically granted under this License. You may not convey a covered
|
||||
work if you are a party to an arrangement with a third party that is
|
||||
in the business of distributing software, under which you make payment
|
||||
to the third party based on the extent of your activity of conveying
|
||||
the work, and under which the third party grants, to any of the
|
||||
parties who would receive the covered work from you, a discriminatory
|
||||
patent license (a) in connection with copies of the covered work
|
||||
conveyed by you (or copies made from those copies), or (b) primarily
|
||||
for and in connection with specific products or compilations that
|
||||
contain the covered work, unless you entered into that arrangement,
|
||||
or that patent license was granted, prior to 28 March 2007.
|
||||
|
||||
Nothing in this License shall be construed as excluding or limiting
|
||||
any implied license or other defenses to infringement that may
|
||||
otherwise be available to you under applicable patent law.
|
||||
|
||||
12. No Surrender of Others' Freedom.
|
||||
|
||||
If conditions are imposed on you (whether by court order, agreement or
|
||||
otherwise) that contradict the conditions of this License, they do not
|
||||
excuse you from the conditions of this License. If you cannot convey a
|
||||
covered work so as to satisfy simultaneously your obligations under this
|
||||
License and any other pertinent obligations, then as a consequence you may
|
||||
not convey it at all. For example, if you agree to terms that obligate you
|
||||
to collect a royalty for further conveying from those to whom you convey
|
||||
the Program, the only way you could satisfy both those terms and this
|
||||
License would be to refrain entirely from conveying the Program.
|
||||
|
||||
13. Use with the GNU Affero General Public License.
|
||||
|
||||
Notwithstanding any other provision of this License, you have
|
||||
permission to link or combine any covered work with a work licensed
|
||||
under version 3 of the GNU Affero General Public License into a single
|
||||
combined work, and to convey the resulting work. The terms of this
|
||||
License will continue to apply to the part which is the covered work,
|
||||
but the special requirements of the GNU Affero General Public License,
|
||||
section 13, concerning interaction through a network will apply to the
|
||||
combination as such.
|
||||
|
||||
14. Revised Versions of this License.
|
||||
|
||||
The Free Software Foundation may publish revised and/or new versions of
|
||||
the GNU General Public License from time to time. Such new versions will
|
||||
be similar in spirit to the present version, but may differ in detail to
|
||||
address new problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the
|
||||
Program specifies that a certain numbered version of the GNU General
|
||||
Public License "or any later version" applies to it, you have the
|
||||
option of following the terms and conditions either of that numbered
|
||||
version or of any later version published by the Free Software
|
||||
Foundation. If the Program does not specify a version number of the
|
||||
GNU General Public License, you may choose any version ever published
|
||||
by the Free Software Foundation.
|
||||
|
||||
If the Program specifies that a proxy can decide which future
|
||||
versions of the GNU General Public License can be used, that proxy's
|
||||
public statement of acceptance of a version permanently authorizes you
|
||||
to choose that version for the Program.
|
||||
|
||||
Later license versions may give you additional or different
|
||||
permissions. However, no additional obligations are imposed on any
|
||||
author or copyright holder as a result of your choosing to follow a
|
||||
later version.
|
||||
|
||||
15. Disclaimer of Warranty.
|
||||
|
||||
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
|
||||
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
|
||||
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
|
||||
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
|
||||
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
|
||||
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
|
||||
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||||
|
||||
16. Limitation of Liability.
|
||||
|
||||
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
|
||||
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
|
||||
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
|
||||
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
|
||||
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
|
||||
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
|
||||
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
|
||||
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
|
||||
SUCH DAMAGES.
|
||||
|
||||
17. Interpretation of Sections 15 and 16.
|
||||
|
||||
If the disclaimer of warranty and limitation of liability provided
|
||||
above cannot be given local legal effect according to their terms,
|
||||
reviewing courts shall apply local law that most closely approximates
|
||||
an absolute waiver of all civil liability in connection with the
|
||||
Program, unless a warranty or assumption of liability accompanies a
|
||||
copy of the Program in return for a fee.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
How to Apply These Terms to Your New Programs
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
If you develop a new program, and you want it to be of the greatest
|
||||
possible use to the public, the best way to achieve this is to make it
|
||||
free software which everyone can redistribute and change under these terms.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
To do so, attach the following notices to the program. It is safest
|
||||
to attach them to the start of each source file to most effectively
|
||||
state the exclusion of warranty; and each file should have at least
|
||||
the "copyright" line and a pointer to where the full notice is found.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
<one line to give the program's name and a brief idea of what it does.>
|
||||
Copyright (C) <year> <name of author>
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
it under the terms of the GNU General Public License as published by
|
||||
the Free Software Foundation, either version 3 of the License, or
|
||||
(at your option) any later version.
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
This program is distributed in the hope that it will be useful,
|
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
GNU General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU General Public License
|
||||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
Also add information on how to contact you by electronic and paper mail.
|
||||
|
||||
If the program does terminal interaction, make it output a short
|
||||
notice like this when it starts in an interactive mode:
|
||||
|
||||
<program> Copyright (C) <year> <name of author>
|
||||
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
|
||||
This is free software, and you are welcome to redistribute it
|
||||
under certain conditions; type `show c' for details.
|
||||
|
||||
The hypothetical commands `show w' and `show c' should show the appropriate
|
||||
parts of the General Public License. Of course, your program's commands
|
||||
might be different; for a GUI interface, you would use an "about box".
|
||||
|
||||
You should also get your employer (if you work as a programmer) or school,
|
||||
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
||||
For more information on this, and how to apply and follow the GNU GPL, see
|
||||
<https://www.gnu.org/licenses/>.
|
||||
|
||||
The GNU General Public License does not permit incorporating your program
|
||||
into proprietary programs. If your program is a subroutine library, you
|
||||
may consider it more useful to permit linking proprietary applications with
|
||||
the library. If this is what you want to do, use the GNU Lesser General
|
||||
Public License instead of this License. But first, please read
|
||||
<https://www.gnu.org/licenses/why-not-lgpl.html>.
|
||||
|
||||
94
MANIFEST.md
Normal file
94
MANIFEST.md
Normal file
@@ -0,0 +1,94 @@
|
||||
## Technical Overview
|
||||
|
||||
### Core Architecture and Memory Management
|
||||
|
||||
The Elyz architecture is built on a strict separation of physical resource ownership and logical resource management, where the kernel acts as a passive arbiter that verifies digital signatures in the form of descriptors. All memory within the system is represented as objects with precise ownership tracking—pages, regions, or segments—and each such object is always tied to a unique owner: a process or the kernel. The owner holds a strong reference, guaranteeing the object's existence until the hierarchical reference counter with per-CPU-core local pools drops to zero, which eliminates cache misses and bus conflicts during metadata updates.
|
||||
|
||||
Support for **revocable capabilities** allows for the forced revocation of rights via descriptor aliases: the kernel instantly invalidates access at the hardware level using Process-Context Identifiers (PCID) without traversing the ownership tree. Processes never operate with direct physical addresses; instead, they work through lightweight descriptor tokens that define access rights and the relationship type (strong, borrowed, or transferred).
|
||||
|
||||
### Distributed Actor Model
|
||||
|
||||
The distributed actor model (replacing a single PM) represents a network of independent PM actors that communicate asynchronously through lock-free message queues. They receive initial memory capital from the bootloader and distribute it without bottlenecks, utilizing **lazy allocation** (on-demand space reservation via page fault) and **ballooning** for the dynamic redistribution of unused memory.
|
||||
|
||||
> **The Asynchrony Tax:** While lock-free queues minimize locking, handling a page fault via asynchronous message passing to a PM actor inherently carries higher latency than a direct, synchronous interrupt handler call in a monolithic Linux kernel.
|
||||
|
||||
### Process Lifecycle and Optimization
|
||||
|
||||
When an application needs to launch, it requests raw memory from PM actors, which carve it into functional objects—thread blocks and descriptor tables. To minimize overhead, an intelligent system call **batching** mechanism is implemented: requests are grouped into queues, reducing the frequency of context switches.
|
||||
|
||||
This batching compromise introduces a baseline delay (**latency floor**) for non-critical tasks: the system appears incredibly smooth in real-time processes (multimedia and input utilize high-priority immediate-execution flags), but "raw" synthetic single-thread computation speed may lag behind Linux.
|
||||
|
||||
### The elibc Layer
|
||||
|
||||
To free developers from manual management, `elibc` is provided as a dynamic shared library (`libelibc.so`) with a standardized POSIX-like ABI based on `musl libc` for full portability. The translation layer inside `elibc` is optimized so that a C programmer sees the familiar `malloc`/`fork` functions, which are transparently converted into native `allocate`/`clone` calls with direct FFI-bindings for Rust/Go, avoiding double context switches.
|
||||
|
||||
When a program requires memory, `elibc` requests a strong descriptor from the Virtual Memory Manager (within the actor model), and the kernel configures the MMU directly after the reference is created, achieving hardware speeds without intermediaries. The results of descriptor verifications are cached in parallel with the TLB, making subsequent accesses practically free.
|
||||
|
||||
### Compatibility Layer (WinServer)
|
||||
|
||||
For a Wine analogue, the key element is the **Personality Server (WinServer)**, which emulates Windows logic within actor isolation, supporting specific WinAPI primitives through dedicated descriptor subsystems. Thanks to descriptors, performance is maximized via borrowing, transferring, and instant revoking: instead of copying data, it performs **zero-copy** transfers of memory rights.
|
||||
|
||||
* **Example:** When rendering a frame, the application transfers the graphics buffer descriptor to the server, or the PM actors project shared memory. Upon timeout or frame completion, access is revoked at the MMU/TLB level, followed by background lazy cleanup.
|
||||
* **Failure Handling:** If a Wine application crashes, the actor-supervision mechanism automatically cascades and annuls all related descriptors, instantly returning resources to the pool and preventing rights fragmentation.
|
||||
|
||||
### Architectural Summary
|
||||
|
||||
In this hierarchy:
|
||||
|
||||
* **The Kernel** is an accountant, verifying links and rights through hardware caches without diving into the logic.
|
||||
* **The Process** is an isolated safe with descriptor-keys stored inside its wallet.
|
||||
* **Wine** is an overseer, translating external instructions into memory bank checks.
|
||||
|
||||
The combination of the actor model, revocable capabilities, and PCID-caching allows for the local "bankruptcy" of any individual component (driver/emulator) without affecting the rest of the system. The balance between performance and convenience is achieved by hiding complex ownership mechanics inside `elibc`, preserving a familiar development style while maintaining architectural reliability at a seL4 level.
|
||||
|
||||
### Focus Mode Feature
|
||||
|
||||
Elyz includes a dedicated feature for manual priority management, allowing either the user or a program (via API access) to determine which actor or process receives more attention. Alternatively, users can enable **"focus mode"**, where only one selected process executes (along with its dependencies and child processes) while everything else "freezes."
|
||||
|
||||
It does not stop; it freezes—once focus mode is deactivated, all operations resume. This serves as a developer tool designed to make compiling exceptionally heavy programs faster and easier directly under Elyz.
|
||||
|
||||
---
|
||||
|
||||
## Philosophy: Nomarchy
|
||||
|
||||
### PART I. Philosophical Pillars (Elyz Axioms)
|
||||
|
||||
#### 1. Separation of Ownership from Management
|
||||
|
||||
In UNIX, the right to perform an action stems from identity (User ID). In Elyz, there are no identities—only assets and rights. A physical resource belongs to whoever created or received it, but logical management is delegated to a distributed network of entities. The system lives in a paradigm of separation of powers: executors (actors) manage, while the arbiter (the kernel) merely stamps digital signatures.
|
||||
|
||||
#### 2. Principle of Total Accounting and Zero Trust
|
||||
|
||||
Every byte of memory, every register, and every quantum of CPU time has a specific, legally bound owner. Security is not a fence around the system; it is the isolation of every single molecule within it. Resources can never be "ownerless" or "shared by default."
|
||||
|
||||
#### 3. Ephemerality and Revocability of Rights
|
||||
|
||||
No granted right is permanent. All trust in Elyz is a temporary loan. The ability to instantly, uncompromisingly, and via cascade annualize any permission at the hardware level is more critical than the procedure used to grant it.
|
||||
|
||||
#### 4. Pragmatic Dualism (Architecture for robots, interface for humans)
|
||||
|
||||
A mathematically sound and strict kernel should not make the programmer suffer. Elyz separates internal chaos from external aesthetics: under the hood lies a highly complex hierarchy of descriptors and asynchronous actors, but externally, the system must present the friendly face of classic POSIX. Rigid security must not compromise ergonomics.
|
||||
|
||||
---
|
||||
|
||||
### PART II. Strict System Rules (Laws of the Kernel and Environment)
|
||||
|
||||
* **Rule 1. The Kernel is a Passive Accountant**
|
||||
The Elyz kernel is deaf and blind to application business logic. It does not know what a "text editor," a "video stream," or a "game" is. Its sole task is to verify the validity of tokens (descriptors), check reference counters, and toggle MMU levers. The kernel never initiates memory management processes on its own—it only executes the will of the actors, provided it is verified by a signature.
|
||||
* **Rule 2. Everything is an Object of Ownership (Replacing UNIX "Everything is a file")**
|
||||
The "everything is a file" concept is obsolete. In Elyz, everything is an object with a strong reference. A thread, a memory region, a page, a descriptor—these are all objects. An object exists precisely as long as its distributed reference counter is alive. When the counter drops to zero, the object is instantly and traceably erased from the physical world by the local CPU core pool.
|
||||
* **Rule 3. Isolation via the Descriptor Wallet**
|
||||
A process has no eyes to see physical memory and no hands to reach it. A process is a blind safe containing a wallet filled with lightweight tokens (descriptors).
|
||||
* *Want to write data?* Show the token.
|
||||
* *Want to transfer data?* Hand over the token (transfer) or let them hold it (borrow).
|
||||
* *Attempting to access a direct address* is a system crime punished by the immediate destruction of the context.
|
||||
|
||||
|
||||
* **Rule 4. Decentralization of Power (Anti-PM)**
|
||||
There is no single point of failure in the system. Instead of a monolithic Process and Memory Manager (as seen in UNIX/Linux), Elyz is governed by a federation of independent, asynchronous PM Actors. They communicate via lock-free message queues. If one actor misbehaves or becomes overloaded, the remaining actors continue carving out memory and distributing starting capital. The local bankruptcy of a component does not lead to the collapse of the state.
|
||||
* **Rule 5. Zero-Copy via Rights Migration**
|
||||
Copying bytes from user space to kernel space or between processes is an archaic overhead. Data must remain stationary. Only the rights to the data move. Passing a graphics frame or a network packet is an instantaneous tectonic shift of descriptors within the MMU/TLB tables. Ownership changes hands at the speed of CPU thought.
|
||||
* **Rule 6. The Batching Compromise (Intelligent Batching)**
|
||||
A system call is an expensive luxury. Elyz forbids processes from poking the kernel for every minor operation. Requests are packed into queues and sent in bulk (batching). However, the system knows the cost of delays: media streams (audio, frames, real-time input) are marked with an "Execute Immediately!" flag, bypassing the general queue.
|
||||
* **Rule 7. Personalities are Simply Isolated Guests**
|
||||
Elyz lacks native support for Windows, Linux, or macOS. Supporting other operating systems does not involve dirty hacks inside the kernel. Instead, isolated Personality Servers (such as WinServer) are used. To Elyz, a foreign OS is just another overseer actor that takes Elyz resources and translates them into a language the guest application understands. If the guest "dies," the actor-supervision mechanism cascades to annualize all its checks, returning memory to the pool without a single drop of fragmentation.
|
||||
626
PLAN.md
Normal file
626
PLAN.md
Normal file
@@ -0,0 +1,626 @@
|
||||
# Elyz (LISA) Microkernel — Полный план оптимизации и полировки
|
||||
|
||||
> Анализ кода по состоянию на июнь 2026.
|
||||
> Составлен на основе исходников: `kernel/src/`, linker-скрипты, Cargo.toml, Makefile.
|
||||
|
||||
---
|
||||
|
||||
## Содержание
|
||||
|
||||
1. [Условные обозначения](#условные-обозначения)
|
||||
2. [Критические баги (не работает)](#ф-1-критические-баги-фаза-1)
|
||||
3. [Средние баги (UB / утечки)](#ф-1-критические-баги-фаза-1)
|
||||
4. [Проблемы архитектуры](#ф-2-архитектурный-рефакторинг-фаза-2)
|
||||
5. [Code Quality & Naming](#ф-3-code-quality--naming-фаза-3)
|
||||
6. [Производительность](#ф-4-производительность-фаза-4)
|
||||
7. [Инфраструктура](#ф-5-инфраструктура-фаза-5)
|
||||
8. [Полный пошаговый план (сортированный)](#полный-пошаговый-план-сортированный-по-приоритету)
|
||||
9. [Roadmap](#roadmap)
|
||||
|
||||
---
|
||||
|
||||
## Условные обозначения
|
||||
|
||||
| Метка | Значение |
|
||||
|-------|----------|
|
||||
| 🔴 CRIT | Системный баг — крах, silent corruption, UB |
|
||||
| 🟡 HIGH | Утечка памяти, логическая ошибка |
|
||||
| 🟢 MED | Неоптимальность, технический долг |
|
||||
| 🔵 LOW | Косметика, нейминг, кодстайл |
|
||||
|
||||
---
|
||||
|
||||
## Ф 1: Критические баги (Фаза 1 — CRIT 1.1–1.6 ✅ исправлены)
|
||||
|
||||
### 🔴 CRIT 1.6 — `tlb_flush_asid` не работает с >64 ядрами ✅ исправлено
|
||||
|
||||
**Файл**: `kernel/src/mem/vmm.rs:794-802`
|
||||
|
||||
**Проблема**: `1u64 << current_core` при `current_core >= 64` — UB (panic в debug, wrap в release).
|
||||
|
||||
**Решение**: Добавлена проверка `current_core < 64` перед shift. Для core >= 64 self-exclusion пропускается (core не входит в нижние 64 бита маски). В `handle_tlb_shootdown_ipi` аналогичная проверка предотвращает запись в маску по некорректному смещению.
|
||||
|
||||
```rust
|
||||
let target_mask = if current_core < 64 {
|
||||
active_cpus & !(1u64 << current_core)
|
||||
} else {
|
||||
active_cpus
|
||||
};
|
||||
```
|
||||
|
||||
**TODO**: При реализации SMP с >64 ядер — перейти на `AtomicU128` или динамический список APIC ID.
|
||||
|
||||
---
|
||||
|
||||
## Ф 1.5: Средние баги (Фаза 1.5 — MED 1.7–1.10 ✅ исправлены)
|
||||
|
||||
### 🟡 MED 1.7 — `unwrap_unchecked()` на Option без full fence ✅ исправлено
|
||||
|
||||
**Файл**: `kernel/src/mem/pm_router.rs:70-82`
|
||||
|
||||
**Проблема**: TOCTOU race между check `is_ready` и `unwrap_unchecked()`. Хотя `is_ready` никогда не сбрасывается, code smell.
|
||||
|
||||
**Решение**: Добавлен комментарий-инвариант, объясняющий почему TOCTOU безопасен (is_ready set-once, never cleared). Panic path вынесен в `#[cold]` функцию:
|
||||
|
||||
```rust
|
||||
#[cold]
|
||||
fn not_initialized() -> ! {
|
||||
panic!("FATAL: PMRouter is accessed before initialization!")
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🟡 MED 1.8 — `process_messages` не имеет квоты ✅ исправлено
|
||||
|
||||
**Файл**: `kernel/src/mem/pm_manages.rs:328-354`
|
||||
|
||||
**Проблема**: Дренирует всю очередь за один вызов. Если 10 000 сообщений, заблокирует ядро на неопределённое время.
|
||||
|
||||
**Решение**: Добавлен лимит `MAX_MESSAGES_PER_CALL = 64`. После обработки 64 сообщений вызов возвращает управление, позволяя другим компонентам ядра получить процессорное время.
|
||||
|
||||
---
|
||||
|
||||
### 🟡 MED 1.9 — Bitmap PMM инициализирует `ref_counts` в 1, потом маркирует used-фреймы ✅ исправлено
|
||||
|
||||
**Файл**: `kernel/src/mem/pmm.rs:49-76`
|
||||
|
||||
**Проблема**: `fill(0xFF)` + `fill(1)` для всех страниц, затем итерация free по USABLE. Лишняя работа.
|
||||
|
||||
**Решение**: Инвертирована логика: `fill(0)` для bitmap и ref_counts, затем итерация по **не-USABLE** записям для mark_used. Убрана лишняя запись в резервированные страницы.
|
||||
|
||||
---
|
||||
|
||||
### 🟡 MED 1.10 — Page fault handler берёт `KERNEL_SPACE.lock()` каждый раз ✅ исправлено
|
||||
|
||||
**Файл**: `kernel/src/cpu/interrupts.rs:109-148`
|
||||
|
||||
**Проблема**: Двойной захват `KERNEL_SPACE.lock()` — в `process_deferred_mmu_events()` и в `rust_page_fault_handler()`. На SMP каждый PF будет сериализован.
|
||||
|
||||
**Решение**: `process_deferred_mmu_events()` встроен в `rust_page_fault_handler` — блокировка захватывается один раз и удерживается для обеих операций. Добавлен TODO о переходе на `RwLock` для SMP.
|
||||
|
||||
---
|
||||
|
||||
## Ф 2: Архитектурный рефакторинг (Фаза 2)
|
||||
|
||||
### 🏗️ ARCH 2.1 — HHDM offset передаётся параметром в каждую функцию
|
||||
|
||||
**Проблема**: `hhdm: u64` тащится через все вызовы:
|
||||
```rust
|
||||
p4.map_region(virt, phys, size, flags, hhdm);
|
||||
p4.map_page(virt, frame, flags, hhdm);
|
||||
```
|
||||
|
||||
Загрязняет сигнатуры и не нужно (HHDM offset — константа после boot).
|
||||
|
||||
**Решение**: Глобальный `static HHDM_OFFSET: AtomicU64`:
|
||||
```rust
|
||||
pub fn init(offset: u64) { HHDM_OFFSET.store(offset, Ordering::Release); }
|
||||
pub fn get() -> u64 { HHDM_OFFSET.load(Ordering::Relaxed) }
|
||||
```
|
||||
|
||||
Убрать `hhdm` из параметров всех функций paging/vmm.
|
||||
|
||||
---
|
||||
|
||||
### 🏗️ ARCH 2.2 — Buddy allocator: O(n) поиск при free()
|
||||
|
||||
**Файл**: `kernel/src/mem/buddy.rs:246`
|
||||
|
||||
```rust
|
||||
if let Some(pos) = self.free_lists[order].iter().position(|&b| b == buddy_idx) {
|
||||
```
|
||||
|
||||
Линейный поиск по `Vec<usize>` — O(количество блоков этого порядка). При 10 000 блоков порядка 0, free() занимает тысячи итераций.
|
||||
|
||||
**Решение**: Перейти на **intrusive linked list** через сами страницы: первые 8 байт неиспользуемой страницы хранят `next` указатель. Даёт O(1) удаление.
|
||||
|
||||
Или хеш-таблицу: `HashMap<usize, ()>` (buddy_idx → exists).
|
||||
|
||||
---
|
||||
|
||||
### 🏗️ ARCH 2.3 — PMM Bitmap: O(n) allocation scan
|
||||
|
||||
**Файл**: `kernel/src/mem/pmm.rs:128-158`
|
||||
|
||||
Побайтовый (побитовый) scan всего bitmap в поиске нулевого бита. На системе с 64 GB RAM → 64M бит → до 8M итераций на аллокацию.
|
||||
|
||||
**Решение**: Использовать **tree bitmap** (buddy-style bitmap). Хранить дополнительный уровень с `u64` словами, где каждый бит = `all_ones(word)`. Позволяет за O(1) найти свободный блок через `ctz` / `clz`.
|
||||
|
||||
---
|
||||
|
||||
### 🏗️ ARCH 2.4 — Channel `wait_for_response` — busy-wait
|
||||
|
||||
**Файл**: `kernel/src/mem/pm_router.rs:130-137`
|
||||
|
||||
```rust
|
||||
while channel.state.load(Ordering::Acquire) != STATE_READY {
|
||||
core::hint::spin_loop();
|
||||
}
|
||||
```
|
||||
|
||||
100% CPU consumption в ожидании. Для реального асинхронного рантайма нужен yield-to-scheduler или waker.
|
||||
|
||||
**Решение**: Добавить `Waker` registration: поток регистрирует свой `Waker` в канале, а `route_responses` вызывает `wake()` при записи результата. Пока планировщика нет — хотя бы HLT-based ожидание.
|
||||
|
||||
---
|
||||
|
||||
### 🏗️ ARCH 2.5 — Capability revocation: O(n) рекурсия
|
||||
|
||||
**Файл**: `kernel/src/cap/mod.rs:97-137`
|
||||
|
||||
`revoke_internal()` сканирует **весь CNode** на каждом уровне рекурсии. С 256+ слотами и глубиной 10 → 2560 итераций.
|
||||
|
||||
**Решение**: Держать per-slot список children:
|
||||
```rust
|
||||
struct CNodeSlot {
|
||||
cap: Capability,
|
||||
parent: Option<usize>,
|
||||
children: Vec<usize>, // явный список наследников
|
||||
}
|
||||
```
|
||||
|
||||
Отзыв — проход по дереву без сканирования всей таблицы.
|
||||
|
||||
---
|
||||
|
||||
### 🏗️ ARCH 2.6 — Нет SMP startup кода
|
||||
|
||||
`ACTIVE_CPUS_MASK = 1` (только BSP). TLB shootdown кидает IPI в пустоту. Нет:
|
||||
- SIPI trampoline page
|
||||
- Startup sequence для AP
|
||||
- Per-CPU data areas
|
||||
- Stack allocation для AP
|
||||
|
||||
**Решение**: Реализовать SIPI-последовательность:
|
||||
1. Выделить страницу по адресу <1MB (real mode trampoline)
|
||||
2. Записать startup код (выход из real mode → long mode → setup stack → jump to rust)
|
||||
3. Отправить SIPI с вектором на эту страницу
|
||||
4. AP просыпается, инициализирует LAPIC, регистрируется в `ACTIVE_CPUS_MASK`
|
||||
|
||||
---
|
||||
|
||||
### 🏗️ ARCH 2.7 — Вся память в HHDM identity-map при старте
|
||||
|
||||
**Файл**: `kernel/src/main.rs:194-201`
|
||||
|
||||
```rust
|
||||
for entry in mmap_res.entries() {
|
||||
p4.map_region(virt_hhdm, phys, entry.length, flags, hhdm_offset);
|
||||
if entry.entry_type != RESERVED {
|
||||
p4.map_region(VirtAddr(entry.base), phys, entry.length, flags, hhdm_offset);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Все *не-reserved* регионы identity-map'ятся по физическому адресу (second mapping). На системе с 64GB RAM и 4KB страницами = 16M calls к `map_page`. Каждый call:
|
||||
- Аллоцирует до 3 страниц таблиц (P3, P2, P1)
|
||||
- INVLPG на каждую 4KB страницу
|
||||
- Время: ~часы при эмуляции, минуты при KVM
|
||||
|
||||
**Решение**: Использовать **2MB huge pages** (`PageTableFlags::HUGE_PAGE`) для identity map. 64GB → 32K entries вместо 16M. INVLPG заменяется на CR3 reload.
|
||||
|
||||
---
|
||||
|
||||
## Ф 3: Code Quality & Naming (Фаза 3)
|
||||
|
||||
### 🧹 CQ 3.1 — Dead code: FramebufferDisplay + Console (main.rs)
|
||||
|
||||
**Файл**: `kernel/src/main.rs:42-144`
|
||||
|
||||
Две структуры:
|
||||
- `FramebufferDisplay` (реализует `DrawTarget` для `embedded-graphics`)
|
||||
- `Console` (обёртка над `FramebufferDisplay` + `embedded-graphics`)
|
||||
|
||||
Никто не используется. Настоящий консольный вывод — `tty::Console`. При этом `embedded-graphics` висит в Cargo.toml как зависимость.
|
||||
|
||||
**Решение**: Удалить об struct + реализацию, убрать `embedded-graphics` из зависимостей.
|
||||
|
||||
---
|
||||
|
||||
### 🧹 CQ 3.2 — Unused imports в main.rs
|
||||
|
||||
**Файл**: `kernel/src/main.rs:6-16`
|
||||
|
||||
```rust
|
||||
use core::fmt::{self, Write};
|
||||
use alloc::vec::Vec;
|
||||
use crate::mem::paging::{PageTable, PageTableFlags};
|
||||
use crate::mem::address::{PhysAddr, VirtAddr};
|
||||
use crate::cap::{Relation, CapRights, Capability, CapObject};
|
||||
use crate::mem::pm_manages::{PMActor, PMRequest, PMResult};
|
||||
```
|
||||
|
||||
Часть из них используется, часть — нет (например `fmt::self`, `PageTable`). После удаления dead code (3.1) ещё часть станет не нужна.
|
||||
|
||||
---
|
||||
|
||||
### 🧹 CQ 3.3 — `#[allow(dead_code)]` на весь mod vmm
|
||||
|
||||
**Файл**: `kernel/src/mem/vmm.rs:27`
|
||||
|
||||
```rust
|
||||
#![allow(dead_code)]
|
||||
```
|
||||
|
||||
Отключает warning для ВСЕГО модуля. 800+ строк кода с отключённой проверкой мёртвого кода. Означает что значительная часть VMM API не используется сейчас — надо либо использовать, либо убрать, либо маркировать `#[allow(dead_code)]` на конкретные единицы.
|
||||
|
||||
---
|
||||
|
||||
### 🧹 CQ 3.4 — Имя проекта: LISA / Elyz / kernel
|
||||
|
||||
| Где | Что написано |
|
||||
|-----|-------------|
|
||||
| `Cargo.toml` | `name = "LISA"` |
|
||||
| `README.md` | "Elyz" |
|
||||
| `limine.conf` | `Elyz (LISA KERNEL)` |
|
||||
| `GNUmakefile` | `elyz-x86_64.iso` |
|
||||
| `main.rs` | `LIS4` (LISA 4?) |
|
||||
|
||||
Три разных публичных имени.
|
||||
|
||||
---
|
||||
|
||||
### 🧹 CQ 3.5 — Странное имя файла: `pm_manages.rs`
|
||||
|
||||
Грамматически некорректно (глагол в имени файла). Основная структура — `PMActor`, файл должен называться `pm_actor.rs`.
|
||||
|
||||
---
|
||||
|
||||
### 🧹 CQ 3.6 — Мёртвые linker-скрипты
|
||||
|
||||
- `linker-aarch64.ld`
|
||||
- `linker-riscv64.ld`
|
||||
- `linker-loongarch64.ld`
|
||||
|
||||
Не используются (build.rs хардкодит `linker-x86_64.ld`). Ни один из этих таргетов не подключен в `rust-toolchain.toml`. Dead weight.
|
||||
|
||||
---
|
||||
|
||||
### 🧹 CQ 3.7 — Locked SpinLock без backoff
|
||||
|
||||
**Файл**: `kernel/src/mem/allocator.rs:27`
|
||||
|
||||
```rust
|
||||
while self.lock.compare_exchange_weak(false, true, Ordering::Acquire, Ordering::Relaxed).is_err() {
|
||||
core::hint::spin_loop();
|
||||
}
|
||||
```
|
||||
|
||||
`spin_loop()` на x86 — это `REP NOP` (PAUSE). Это правильно, но при длительном ожидании нужно экспоненциально увеличивать задержку.
|
||||
|
||||
---
|
||||
|
||||
### 🧹 CQ 3.8 — `log!` максрос не lock-free
|
||||
|
||||
При вызове `info!(...)` внутри макроса:
|
||||
1. Вызывается `SerialPort::init()` (см. CRIT 1.3)
|
||||
2. Пишет на экран через framebuffer
|
||||
3. Пишет в serial
|
||||
|
||||
Если в PF handler вызвать `info!`, и PF произошёл из-за framebuffer DMA или serial IRQ — будет reentrancy. Надо маркировать как `#[inline(never)]` или добавить reentrancy guard.
|
||||
|
||||
---
|
||||
|
||||
## Ф 4: Производительность (Фаза 4)
|
||||
|
||||
### ⚡ PERF 4.1 — Bitmap PMM: word-level skip
|
||||
|
||||
**Файл**: `kernel/src/mem/pmm.rs:139`
|
||||
|
||||
```rust
|
||||
if self.bitmap[byte_idx] == 0xFF { continue; }
|
||||
```
|
||||
|
||||
Проверка на уровне байта — можно на уровне `u64` слова: `if word == u64::MAX { skip 8 bytes }`. В 8x меньше итераций.
|
||||
|
||||
---
|
||||
|
||||
### ⚡ PERF 4.2 — Buddy free: O(n) → O(1)
|
||||
|
||||
**Файл**: `kernel/src/mem/buddy.rs:246`
|
||||
|
||||
См. ARCH 2.2. Замена `Vec` на intrusive list — ключевой перф-фикс.
|
||||
|
||||
---
|
||||
|
||||
### ⚡ PERF 4.4 — INVLPG в bulk_map: batch flush
|
||||
|
||||
**Файл**: `kernel/src/mem/paging.rs:119-121`
|
||||
|
||||
```rust
|
||||
unsafe {
|
||||
asm!("invlpg [{}]", in(reg) virt.0, ...);
|
||||
}
|
||||
```
|
||||
|
||||
`map_region` вызывает `map_page` в цикле — каждая итерация делает INVLPG. 16M страниц = 16M INVLPG. CPU заливает TLBs и потом их же сбрасывает.
|
||||
|
||||
**Решение**: Убрать INVLPG из `map_page` (оставить вызывающему), сделать отдельную функцию `flush_range(virt, pages)` или вызов `invpcid` type-2 (flush-by-range).
|
||||
|
||||
**Но осторожно**: INVLPG нужен при модификации активной таблицы. Если identity-map ещё не загружена — можно без INVLPG.
|
||||
|
||||
---
|
||||
|
||||
### ⚡ PERF 4.5 — TLB shootdown: busy-wait → IPI + HLT
|
||||
|
||||
**Файл**: `kernel/src/mem/vmm.rs:809-811`
|
||||
|
||||
```rust
|
||||
while SHOOTDOWN_ACK.load(Ordering::Acquire) & target_mask != target_mask {
|
||||
spin_loop();
|
||||
}
|
||||
```
|
||||
|
||||
Пока нет AP — код мёртвый. Когда AP появятся, надо чтобы AP висели в `HLT` и просыпались от IPI с TLB_SHOOTDOWN_VECTOR.
|
||||
|
||||
---
|
||||
|
||||
### ⚡ PERF 4.6 — Page fault handler: lock-free VMA tree
|
||||
|
||||
**Файл**: `kernel/src/cpu/interrupts.rs:110`
|
||||
|
||||
`KERNEL_SPACE.lock()` — каждый PF сериализуется. На одноядерной системе — OK, на SMP — bottleneck.
|
||||
|
||||
**Решение**: `RWLock` на VMA region или lock-free RB-tree для VMA поиска.
|
||||
|
||||
---
|
||||
|
||||
## Ф 5: Инфраструктура (Фаза 5)
|
||||
|
||||
### 🛠️ INFRA 5.1 — SMP AP Startup Code
|
||||
|
||||
Отсутствует. Нужен:
|
||||
- Startup trampoline в real mode (<1MB)
|
||||
- Переход в protected → long mode
|
||||
- Настройка GDT/IDT для AP
|
||||
- Per-CPU stack
|
||||
- Регистрация в `ACTIVE_CPUS_MASK`
|
||||
|
||||
---
|
||||
|
||||
### 🛠️ INFRA 5.2 — Мультиархитектурность
|
||||
|
||||
Есть linker-скрипты под aarch64, riscv64, loongarch64, но:
|
||||
- `build.rs` хардкодит `linker-x86_64.ld`
|
||||
- ASM в interrupts.rs только x86
|
||||
- LAPIC — x86-specific
|
||||
- Port IO — x86-specific
|
||||
- IDT — x86-specific
|
||||
|
||||
Для портирования нужна архитектурная абстракция — traits:
|
||||
```rust
|
||||
trait Arch {
|
||||
type PteFlags;
|
||||
fn init_interrupts();
|
||||
fn send_ipi(core_id: u32, vector: u8);
|
||||
fn read_cycle_counter() -> u64;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🛠️ INFRA 5.3 — Unit-тесты для Buddy Allocator
|
||||
|
||||
Buddy allocator — идеальный кандидат для `#[cfg(test)]`:
|
||||
- Проверка инвариантов после alloc/free sequence
|
||||
- Fuzzing случайных порядков
|
||||
- Проверка coalescing (граничные случаи)
|
||||
- Проверка alignment
|
||||
|
||||
---
|
||||
|
||||
### 🛠️ INFRA 5.4 — Утилита для бинарного патчинга
|
||||
|
||||
Сейчас результирующий ELF надо руками objcopy'ить. Можно добавить Cargo alias или build.rs скрипт для автоматизации.
|
||||
|
||||
---
|
||||
|
||||
### 🛠️ INFRA 5.5 — GDB/QEMU debugging support
|
||||
|
||||
Добавить `.gdbinit` / `gdb.py` с:
|
||||
- `target remote localhost:1234`
|
||||
- Разгрузка символов из ELF
|
||||
- Макросы для просмотра page table, capability, VMA regions
|
||||
|
||||
---
|
||||
|
||||
## Ф 6: Безопасность (сквозная)
|
||||
|
||||
### 🛡️ SEC 6.1 — Capability minting не проверяет `Relation` parent
|
||||
|
||||
**Файл**: `kernel/src/cap/mod.rs:64-77`
|
||||
|
||||
```rust
|
||||
if !src_slot.cap.rights.contains(CapRights::GRANT) {
|
||||
return Err("Insufficient rights to mint (Missing GRANT flag)");
|
||||
}
|
||||
```
|
||||
|
||||
Проверяет только GRANT флаг. Но не проверяет `relation`: если source — `Borrow`, можно ли mint? В текущей реализации — можно. Должен быть check:
|
||||
|
||||
```rust
|
||||
if src_slot.cap.relation == Relation::Borrow {
|
||||
return Err("Cannot mint from borrowed capability");
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🛡️ SEC 6.2 — CapGuardToken не реализован
|
||||
|
||||
В дескрипторе есть упоминание `guards: Vec<CapGuardToken>`, `Borrow` relation и guard-токены, логика самих guard'ов не реализована. Если есть borrow, mint не должен позволять дальнейшее распространение — но это не проверяется.
|
||||
|
||||
---
|
||||
|
||||
### 🛡️ SEC 6.3 — Нет проверки прав на операции с PMM через capabilities
|
||||
|
||||
`BitmapPMM` полностью глобальный: любой код может вызвать `alloc_frame()`, `free_frame()` без проверки capability. Капы пока хранят токены и права, но PMM их не проверяет.
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Полный пошаговый план (сортированный по приоритету)
|
||||
|
||||
### Этап 1: Исправить критические баги (сейчас — не работает)
|
||||
|
||||
```
|
||||
[X] 1. ✅ HIGH — Исправить target_mask для >64 core
|
||||
Добавлена проверка current_core < 64 перед shift.
|
||||
Estimate: ~10 мин
|
||||
|
||||
[ ] 2. 🔥 HIGH — Исправить ref_counts инициализацию
|
||||
Вместо fill(1) + free, сделать fill(0) + mark_used.
|
||||
Estimate: ~15 мин
|
||||
```
|
||||
|
||||
### Этап 2: Архитектурный рефакторинг
|
||||
|
||||
```
|
||||
[ ] 3. 🏗️ ARCH — Глобальный HHDM_OFFSET
|
||||
AtomicU64 static, убрать hhdm из сигнатур.
|
||||
Estimate: ~1 ч
|
||||
|
||||
[ ] 4. 🏗️ ARCH — Intrusive free-list в buddy allocator
|
||||
O(1) free вместо O(n).
|
||||
Estimate: ~2 ч
|
||||
|
||||
[ ] 5. 🏗️ ARCH — Tree bitmap для PMM
|
||||
word-level scan с ctz.
|
||||
Estimate: ~1.5 ч
|
||||
|
||||
[ ] 6. 🏗️ ARCH — HLT-based wait в router channel
|
||||
Пока нет планировщика — HLT вместо spin.
|
||||
Estimate: ~30 мин
|
||||
|
||||
[ ] 7. 🏗️ ARCH — Capability revoke без O(n) scan
|
||||
Children list в слоте.
|
||||
Estimate: ~1 ч
|
||||
|
||||
[ ] 8. 🏗️ ARCH — Identity-map через 2MB huge pages
|
||||
map_region с HUGE_PAGE флагом для identity.
|
||||
Estimate: ~1 ч
|
||||
```
|
||||
|
||||
### Этап 3: Code Quality
|
||||
|
||||
```
|
||||
[ ] 9. 🧹 CQ — Удалить dead code (FramebufferDisplay, Console, embedded-graphics)
|
||||
+ неиспользуемые импорты.
|
||||
Estimate: ~20 мин
|
||||
|
||||
[ ] 10. 🧹 CQ — Убрать #![allow(dead_code)] из vmm.rs
|
||||
Маркировать конкретные единицы.
|
||||
Estimate: ~15 мин
|
||||
|
||||
[ ] 11. 🧹 CQ — Удалить мёртвые linker-скрипты
|
||||
aarch64/riscv64/loongarch64.ld.
|
||||
Estimate: ~5 мин
|
||||
|
||||
[ ] 12. 🧹 CQ — Переименовать pm_manages.rs → pm_actor.rs
|
||||
+ все референсы.
|
||||
Estimate: ~10 мин
|
||||
|
||||
[ ] 13. 🧹 CQ — Вынести panic.rs из main.rs
|
||||
Отдельный файл для panic_handler.
|
||||
Estimate: ~10 мин
|
||||
|
||||
[ ] 14. 🧹 CQ — Синхронизировать имена: LISA → Elyz
|
||||
Cargo.toml, main.rs ("LIS4"), README.
|
||||
Estimate: ~15 мин
|
||||
|
||||
[ ] 15. 🧹 CQ — Locked spinlock с exponential backoff
|
||||
PAUSE + цикл задержки.
|
||||
Estimate: ~15 мин
|
||||
```
|
||||
|
||||
### Этап 4: Производительность
|
||||
|
||||
```
|
||||
[ ] 16. ⚡ PERF — Bitmap scan: word-level (u64) вместо byte-level
|
||||
+ ctzl для поиска бита.
|
||||
Estimate: ~30 мин
|
||||
|
||||
[ ] 17. ⚡ PERF — INVLPG-free bulk map
|
||||
Убрать INVLPG из map_page, вызов flush_range после.
|
||||
Estimate: ~30 мин
|
||||
|
||||
[ ] 18. ⚡ PERF — process_messages quota
|
||||
Max N сообщений за вызов.
|
||||
Estimate: ~15 мин
|
||||
```
|
||||
|
||||
### Этап 5: Инфраструктура
|
||||
|
||||
```
|
||||
[ ] 19. 🛠️ INFRA — SMP AP startup code
|
||||
SIPI → trampoline → long mode → Rust.
|
||||
Estimate: ~1 неделя
|
||||
|
||||
[ ] 20. 🛠️ INFRA — Unit tests для buddy allocator
|
||||
#[cfg(test)] модуль.
|
||||
Estimate: ~1 ч
|
||||
|
||||
[ ] 21. 🛠️ INFRA — GDB script + QEMU -s
|
||||
.gdbinit для отладки.
|
||||
Estimate: ~30 мин
|
||||
|
||||
[ ] 22. 🛠️ INFRA — Arch trait для x86_64
|
||||
Подготовка к портированию.
|
||||
Estimate: ~3-4 ч
|
||||
```
|
||||
|
||||
### Этап 6: Безопасность
|
||||
|
||||
```
|
||||
[ ] 23. 🛡️ SEC — Проверка Relation при mint
|
||||
Запретить mint из Borrow.
|
||||
Estimate: ~10 мин
|
||||
|
||||
[ ] 24. 🛡️ SEC — CapGuard реализация
|
||||
Привязка guard-токенов к source slot при mint.
|
||||
Estimate: ~45 мин
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Roadmap (визуально)
|
||||
|
||||
```
|
||||
Неделя 1 Неделя 2 Неделя 3
|
||||
┌─────────────────────────┐ ┌─────────────────────────┐ ┌─────────────────────────┐
|
||||
│ ✅ CRIT 1–6 готово │▸▸ │ ARCH 3 HHDM global │▸▸ │ PERF 16 word-level PMM │
|
||||
│ ✅ MED 1.7–1.10 │ │ ARCH 4 intrusive list │ │ PERF 17 INVLPG batch │
|
||||
│ HIGH 2 ref_counts │ │ ARCH 5 tree bitmap │ │ INFRA 19 SMP startup │
|
||||
│ │ │ CQ 9 clean dead code │ │ INFRA 20 unit tests │
|
||||
│ │ │ CQ 11-15 rename/refact│ │ SEC 23-24 caps │
|
||||
└─────────────────────────┘ └─────────────────────────┘ └─────────────────────────┘
|
||||
↓ ↓ ↓
|
||||
Kernel bootable и Код читаемый, Производительность:
|
||||
логирует без крашей архитектура чище SMP + оптимизации
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Документ создан 29 июня 2026 на основе анализа исходного кода проекта Elyz.*
|
||||
63
README.md
63
README.md
@@ -104,50 +104,49 @@ make run KARCH=aarch64
|
||||
## Current Functionality
|
||||
|
||||
| Component | Status | Description |
|
||||
| -------------------- | ------- | -------------------------- |
|
||||
| --- | --- | --- |
|
||||
| Limine Bootloader | ✅ Done | BIOS/UEFI support |
|
||||
| Physical MM (Bitmap) | ✅ Done | Page bitmap |
|
||||
| Page Tables (x86_64) | ✅ Done | 4-level addressing |
|
||||
| Global Allocator | ✅ Done | Bump allocator (8MB) |
|
||||
| Framebuffer | ✅ Done | Graphical output |
|
||||
| Console | ✅ Done | Text output with scrolling |
|
||||
| Physical MM (Bitmap) | ✅ Done | Initial physical frame management |
|
||||
| Page Tables (x86_64) | ✅ Done | 4-level paging, HHDM support |
|
||||
| Kernel Heap | ✅ Done | Slab Allocator (8MB) |
|
||||
| Framebuffer/Console | ✅ Done | Graphical output with font rendering |
|
||||
| PMActor Subsystem | ✅ Done | Distributed memory ownership model |
|
||||
| PMRouter | ✅ Done | Lock-free MPSC response routing |
|
||||
| Capability Model | ✅ Done | Base `CNode` & `Capability` primitives |
|
||||
|
||||
## Memory System Development Status
|
||||
|
||||
| Task | Status | Note |
|
||||
| --------------------------- | -------------- | ----------------------- |
|
||||
| **Basic initialization** | ✅ Ready | Limine, MMU, PMM |
|
||||
| **Physical allocator** | ✅ Ready | BitmapPMM working |
|
||||
| **Paging** | ✅ Ready | P4->P1 mapping |
|
||||
| **Global heap** | ✅ Ready | 8MB bump allocator |
|
||||
| **Descriptor memory model** | ⏳ Not Started | Planned |
|
||||
| **Processes and spaces** | ⏳ Not Started | No IPC |
|
||||
| **Memory syscalls** | ⏳ Not Started | allocate/clone/transfer |
|
||||
| **Weak references** | ⏳ Not Started | borrow/release |
|
||||
| **Process auto-cleanup** | ⏳ Not Started | On exit |
|
||||
| --- | --- | --- |
|
||||
| **Basic Initialization** | ✅ Ready | Boot, MMU, IDT, PMM |
|
||||
| **Buddy Allocator** | ✅ Ready | Efficient per-actor memory carving |
|
||||
| **Async PM Messaging** | ✅ Ready | Actor-based allocation requests |
|
||||
| **Lock-free Routing** | ✅ Ready | Zero-copy response dispatching |
|
||||
| **Descriptor Model** | ⏳ In Progress | `MemoryObject` & refcounting logic |
|
||||
| **Process Isolation** | ⏳ Not Started | Address space cloning/switching |
|
||||
| **Memory Syscalls** | ⏳ Not Started | `allocate`, `transfer`, `grant` |
|
||||
| **Weak References** | ⏳ Not Started | `borrow`/`release` for capabilities |
|
||||
|
||||
## Development Plan (TODO)
|
||||
|
||||
| # | Task | Subtasks | Priority | Dependencies |
|
||||
|---|------|----------|----------|--------------|
|
||||
| 1 | **Descriptor memory model** | | High | PMM, VMM |
|
||||
| | | 1.1. `MemoryObject` structure | | |
|
||||
| | | 1.2. Descriptor table per process | | |
|
||||
| | | 1.3. Refcounting (strong/weak) | | |
|
||||
| | | 1.4. Syscalls: `allocate`, `clone` | | |
|
||||
| 2 | **Processes** | | High | Descriptors |
|
||||
| | | 2.1. PCB structure | | |
|
||||
| | | 2.2. Descriptor Table per process | | |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 1 | **Capabilities System** | | High | PMActor |
|
||||
| | | 1.1. Capability refcounting | | |
|
||||
| | | 1.2. Inter-process delegation | | |
|
||||
| 2 | **Process Management** | | High | Capabilities |
|
||||
| | | 2.1. PCB & Context switching | | |
|
||||
| | | 2.2. Address Space isolation | | |
|
||||
| | | 2.3. Auto-cleanup on exit | | |
|
||||
| 3 | **Syscalls API** | | Medium | Processes |
|
||||
| | | 3.1. `transfer(desc, target)` | | |
|
||||
| | | 3.2. `borrow(desc, duration)` | | |
|
||||
| | | 3.3. `release(desc)` | | |
|
||||
| 4 | **MMU Integration** | | Medium | Syscalls |
|
||||
| | | 4.1. Page table per process | | |
|
||||
| | | 4.2. Mapping by descriptors | | |
|
||||
| | | 4.3. Unmapping on release | | |
|
||||
| 5 | **Circular references** | | Low | All above |
|
||||
| | | 3.2. `mint(desc)` (delegation) | | |
|
||||
| 4 | **Advanced MMU** | | Medium | Syscalls |
|
||||
| | | 4.1. Lazy paging / Copy-on-Write | | |
|
||||
| | | 4.2. TLB Shootdown (SMP support) | | |
|
||||
| 5 | **Security/Hardening** | | Low | All above |
|
||||
| | | 5.1. Circular ref detection | | |
|
||||
| | | 5.2. Capability revocation depth | | |
|
||||
|
||||
## Debugging
|
||||
|
||||
|
||||
152
kernel/docs/boot/introduction.md
Normal file
152
kernel/docs/boot/introduction.md
Normal file
@@ -0,0 +1,152 @@
|
||||
# Процесс загрузки: концептуальная модель
|
||||
|
||||
## Цепочка загрузки
|
||||
|
||||
```
|
||||
Питание включено
|
||||
│
|
||||
▼
|
||||
CPU reset vector (0xFFFFFFF0)
|
||||
│
|
||||
▼
|
||||
UEFI firmware / BIOS
|
||||
│
|
||||
▼
|
||||
Limine bootloader ─────────────────────────────┐
|
||||
│ │
|
||||
│ 1. Переводит CPU в 64-bit long mode │
|
||||
│ 2. Настраивает page tables (identity map) │
|
||||
│ 3. Загружает ядро по физическому адресу │
|
||||
│ 4. Настраивает HHDM │
|
||||
│ 5. Заполняет Limine requests │
|
||||
│ 6. Передаёт управление на kmain │
|
||||
│ │
|
||||
▼ │
|
||||
kmain() (точка входа) │
|
||||
│ │
|
||||
... инициализация ... │
|
||||
│ │
|
||||
▼ │
|
||||
HCF (останов CPU) ─────────────────────────────┘
|
||||
```
|
||||
|
||||
## Limine Boot Protocol
|
||||
|
||||
Elyz использует [Limine](https://github.com/limine-bootloader/limine) —
|
||||
современный bootloader с открытым исходным кодом.
|
||||
|
||||
Статические `Limine requests` сообщают bootloader'у, что нужно ядру:
|
||||
|
||||
| Request | Назначение |
|
||||
|---------|------------|
|
||||
| `FramebufferRequest` | Получить framebuffer для графического вывода |
|
||||
| `MemoryMapRequest` | Получить карту физической памяти |
|
||||
| `HhdmRequest` | Получить HHDM offset |
|
||||
| `ExecutableAddressRequest` | Получить физический/виртуальный адрес ядра |
|
||||
| `BaseRevision` | Проверить совместимость с Limine |
|
||||
|
||||
### Размещение в секциях
|
||||
|
||||
Requests размещаются между специальными маркерами в секции `.data`:
|
||||
|
||||
```rust
|
||||
#[used]
|
||||
#[unsafe(link_section = ".requests_start_marker")]
|
||||
static _START_MARKER: RequestsStartMarker = RequestsStartMarker::new();
|
||||
|
||||
// ... все requests ...
|
||||
|
||||
#[used]
|
||||
#[unsafe(link_section = ".requests_end_marker")]
|
||||
static _END_MARKER: RequestsEndMarker = RequestsEndMarker::new();
|
||||
```
|
||||
|
||||
Линкер скрипт сохраняет эти секции:
|
||||
```ld
|
||||
.data : {
|
||||
*(.data .data.*)
|
||||
KEEP(*(.requests_start_marker))
|
||||
KEEP(*(.requests))
|
||||
KEEP(*(.requests_end_marker))
|
||||
} :data
|
||||
```
|
||||
|
||||
## Точка входа — kmain
|
||||
|
||||
Линкер скрипт: `ENTRY(kmain)`.
|
||||
|
||||
```rust
|
||||
#[unsafe(no_mangle)]
|
||||
unsafe extern "C" fn kmain() -> ! { ... }
|
||||
```
|
||||
|
||||
Атрибуты:
|
||||
- `no_mangle` — сохраняет имя `kmain` (линкер ищет именно его).
|
||||
- `extern "C"` — C ABI (Linux x86-64 calling convention: RDI, RSI, ...).
|
||||
- `unsafe` — на этапе инициализации все операции потенциально опасны.
|
||||
- `-> !` — kmain никогда не возвращается (HALT).
|
||||
|
||||
## Build system
|
||||
|
||||
### GNUmakefile
|
||||
|
||||
```makefile
|
||||
KARCH ?= x86_64
|
||||
RUST_TARGET ?= $(KARCH)-unknown-none
|
||||
RUST_PROFILE ?= dev
|
||||
|
||||
all:
|
||||
RUSTFLAGS="-C relocation-model=static" \
|
||||
cargo build --target $(RUST_TARGET) --profile $(RUST_PROFILE)
|
||||
cp target/$(RUST_TARGET)/$(RUST_PROFILE_SUBDIR)/kernel .
|
||||
```
|
||||
|
||||
- `relocation-model=static`: ядро не использует динамическую перелокацию.
|
||||
- `--target x86_64-unknown-none`: bare-metal target (без ОС).
|
||||
- Результат копируется в `kernel/` (корень).
|
||||
|
||||
### build.rs — linker script
|
||||
|
||||
```rust
|
||||
fn main() {
|
||||
let arch = std::env::var("CARGO_CFG_TARGET_ARCH").unwrap();
|
||||
println!("cargo:rustc-link-arg=-Tlinker-{arch}.ld");
|
||||
println!("cargo:rerun-if-changed=linker-{arch}.ld");
|
||||
}
|
||||
```
|
||||
|
||||
Подставляет правильный linker script для архитектуры.
|
||||
|
||||
### rust-toolchain.toml
|
||||
|
||||
```toml
|
||||
[toolchain]
|
||||
channel = "nightly"
|
||||
targets = ["x86_64-unknown-none"]
|
||||
```
|
||||
|
||||
Требуется nightly Rust из-за:
|
||||
- `#![no_std]`, `#![no_main]`.
|
||||
- `core::arch::global_asm!`, `core::arch::asm!`.
|
||||
- `const { ... }` в инициализации констант.
|
||||
|
||||
## Память: расположение после загрузки
|
||||
|
||||
```
|
||||
Физическая память:
|
||||
┌───────────────────────┐ 0x0
|
||||
│ Reserved │
|
||||
├───────────────────────┤
|
||||
│ Usable (free) │ ← входит в mmap entries
|
||||
├───────────────────────┤
|
||||
│ Kernel image │ ← загружен Limine
|
||||
├───────────────────────┤
|
||||
│ PMM metadata │ ← bitmap + ref_counts + L1
|
||||
├───────────────────────┤
|
||||
│ ... (usable) │
|
||||
└───────────────────────┘ max_addr
|
||||
|
||||
Виртуальная память (Higher Half):
|
||||
0xFFFF_8000_0000_0000 ─── HHDM (вся физическая память 1:1)
|
||||
0xFFFF_9000_0000_0000 ─── Kernel heap (8 MiB)
|
||||
```
|
||||
151
kernel/docs/boot/linker.md
Normal file
151
kernel/docs/boot/linker.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# Линкер-скрипты: архитектурные детали
|
||||
|
||||
## Назначение
|
||||
|
||||
Линкер-скрипты управляют расположением секций ELF-образа ядра.
|
||||
Для каждой архитектуры — свой скрипт, но все они следуют одной схеме.
|
||||
|
||||
## Общая структура
|
||||
|
||||
```
|
||||
OUTPUT_FORMAT(...) ← формат ELF (зависит от архитектуры)
|
||||
ENTRY(kmain) ← точка входа
|
||||
|
||||
PHDRS ← сегменты (program headers)
|
||||
{
|
||||
text PT_LOAD;
|
||||
rodata PT_LOAD;
|
||||
data PT_LOAD;
|
||||
}
|
||||
|
||||
SECTIONS ← расположение секций
|
||||
{
|
||||
. = 0xffffffff80000000; ← база higher half
|
||||
|
||||
.text : { *(.text .text.*) } :text
|
||||
|
||||
. = ALIGN(MAXPAGESIZE);
|
||||
.rodata : { *(.rodata .rodata.*) } :rodata
|
||||
|
||||
. = ALIGN(MAXPAGESIZE);
|
||||
.data : {
|
||||
*(.data .data.*)
|
||||
KEEP(*(.requests_start_marker))
|
||||
KEEP(*(.requests))
|
||||
KEEP(*(.requests_end_marker))
|
||||
} :data
|
||||
|
||||
.bss : { *(.bss .bss.*) *(COMMON) } :data
|
||||
|
||||
/DISCARD/ : { *(.eh_frame*) *(.note .note.*) }
|
||||
}
|
||||
```
|
||||
|
||||
## Детали
|
||||
|
||||
### OUTPUT_FORMAT
|
||||
|
||||
| Архитектура | Формат |
|
||||
|-------------|--------|
|
||||
| x86_64 | `elf64-x86-64` |
|
||||
| aarch64 | `elf64-littleaarch64` |
|
||||
| riscv64 | `elf64-littleriscv` |
|
||||
| loongarch64 | `elf64-loongarch` |
|
||||
|
||||
### Базовый адрес: 0xFFFFFFFF80000000
|
||||
|
||||
Ядро размещается в **higher half** — верхней 2 GiB виртуального
|
||||
адресного пространства. Это стандартная практика для x86-64:
|
||||
|
||||
```
|
||||
0x0000_0000_0000_00000 ─── user space (не используется ядром)
|
||||
0xFFFF_8000_0000_00000 ─── kernel space (higher half)
|
||||
0xFFFF_FFFF_FFFF_FFFF ─── конец
|
||||
```
|
||||
|
||||
Любой адрес в области 0xFFFF800000000000 — 0xFFFFFFFFFFFFFFFF корректен;
|
||||
0xFFFFFFFF80000000 выбран как начало typical higher half региона.
|
||||
|
||||
### PHDRS: PT_LOAD сегменты
|
||||
|
||||
Bootloader загружает только PT_LOAD сегменты. Их три:
|
||||
|
||||
1. **text**: код + inline-константы.
|
||||
2. **rodata**: неизменяемые данные (строки, таблицы).
|
||||
3. **data**: изменяемые данные + BSS.
|
||||
|
||||
### Section alignment
|
||||
|
||||
```ld
|
||||
. = ALIGN(CONSTANT(MAXPAGESIZE));
|
||||
```
|
||||
|
||||
MAXPAGESIZE = 0x1000 (4 KiB). Каждая секция начинается с новой страницы,
|
||||
что даёт bootloader'у правильные MMU permissionы:
|
||||
- text = read + execute (no write)
|
||||
- rodata = read (no write, no execute)
|
||||
- data = read + write (no execute)
|
||||
|
||||
### Limine Requests в .data
|
||||
|
||||
```ld
|
||||
KEEP(*(.requests_start_marker))
|
||||
KEEP(*(.requests))
|
||||
KEEP(*(.requests_end_marker))
|
||||
```
|
||||
|
||||
- `KEEP` — запрещает линкеру выбрасывать эти секции (dead code elimination).
|
||||
- `.requests_start_marker` и `.requests_end_marker` — маркеры границ.
|
||||
- Bootloader сканирует память между ними, чтобы найти requests.
|
||||
|
||||
### BSS
|
||||
|
||||
```ld
|
||||
.bss : {
|
||||
*(.bss .bss.*)
|
||||
*(COMMON)
|
||||
} :data
|
||||
```
|
||||
|
||||
- BSS — неинициализированные глобальные переменные.
|
||||
- Занимает место в виртуальной памяти, но не в ELF-файле.
|
||||
- Bootloader обнуляет BSS перед передачей управления.
|
||||
|
||||
### DISCARD
|
||||
|
||||
```ld
|
||||
/DISCARD/ : {
|
||||
*(.eh_frame*)
|
||||
*(.note .note.*)
|
||||
}
|
||||
```
|
||||
|
||||
- `.eh_frame*` — исключительные фреймы C++/Rust unwinding.
|
||||
- `.note.*` — ELF notes.
|
||||
- Не нужны bare-metal ядру, могут вызвать проблемы.
|
||||
|
||||
## Архитектурные различия
|
||||
|
||||
### RISC-V
|
||||
|
||||
```ld
|
||||
.data : {
|
||||
*(.data .data.*)
|
||||
KEEP(*(.requests_start_marker))
|
||||
KEEP(*(.requests))
|
||||
KEEP(*(.requests_end_marker))
|
||||
*(.sdata .sdata.*) ← RISC-V: small data
|
||||
} :data
|
||||
|
||||
.bss : {
|
||||
*(.sbss .sbss.*) ← RISC-V: small BSS
|
||||
*(.bss .bss.*)
|
||||
*(COMMON)
|
||||
} :data
|
||||
```
|
||||
|
||||
RISC-V имеет `.sdata`/`.sbss` секции для small data (GP-relative addressing).
|
||||
|
||||
### AArch64 и LoongArch64
|
||||
|
||||
Идентичны x86_64, за исключением OUTPUT_FORMAT.
|
||||
146
kernel/docs/capability/cnode.md
Normal file
146
kernel/docs/capability/cnode.md
Normal file
@@ -0,0 +1,146 @@
|
||||
# CNode: таблица capability: `mod.rs`
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
**CNode** (Capability Node) — это массив слотов, каждый из которых
|
||||
может хранить один capability. Это аналог файловой таблицы в Unix,
|
||||
но для capabilities.
|
||||
|
||||
```
|
||||
CNode {
|
||||
slots: Vec<Locked<CNodeSlot>>
|
||||
}
|
||||
|
||||
CNodeSlot {
|
||||
cap: Capability,
|
||||
parent_idx: Option<usize>, // индекс родительского слота
|
||||
}
|
||||
```
|
||||
|
||||
## Инициализация
|
||||
|
||||
```rust
|
||||
pub fn new(size: usize) -> Self {
|
||||
let mut slots = Vec::with_capacity(size);
|
||||
for _ in 0..size {
|
||||
slots.push(Locked::new(CNodeSlot {
|
||||
cap: Capability::empty(),
|
||||
parent_idx: None,
|
||||
}));
|
||||
}
|
||||
Self { slots }
|
||||
}
|
||||
```
|
||||
|
||||
Все слоты изначально пустые (`Capability::empty()`).
|
||||
|
||||
## Операции
|
||||
|
||||
### insert(slot, cap) — вставка
|
||||
|
||||
```rust
|
||||
pub fn insert(&self, slot: usize, cap: Capability) -> Result<(), &'static str> {
|
||||
if slot >= self.slots.len() { return Err("Index out of bounds"); }
|
||||
let mut s = self.slots[slot].lock();
|
||||
s.cap = cap;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
Простая вставка без проверки (перезаписывает существующий).
|
||||
|
||||
### mint(src, dest, relation, rights) — создание потомка
|
||||
|
||||
```rust
|
||||
pub fn mint(&self, src: usize, dest: usize, relation: Relation, rights: CapRights) -> Result<(), &'static str>
|
||||
```
|
||||
|
||||
**Валидация:**
|
||||
- src и dest в пределах массива.
|
||||
- src != dest (дедлок не имеет смысла, но блокировка была бы корректна).
|
||||
- src содержит валидный capability.
|
||||
- src имеет GRANT.
|
||||
|
||||
**Процесс:**
|
||||
1. Захват блокировок в порядке возрастания индекса (lock ranking).
|
||||
2. Вычисление `final_rights = src.rights & rights`.
|
||||
3. Копирование Capability в dest с новыми правами и relation.
|
||||
4. Установка `parent_idx = Some(src)`.
|
||||
|
||||
### revoke(slot_idx) — отзыв
|
||||
|
||||
```rust
|
||||
pub fn revoke(&self, slot_idx: usize) -> Result<(), &'static str>
|
||||
```
|
||||
|
||||
Каскадное удаление:
|
||||
|
||||
```
|
||||
revoke_internal(slot_idx):
|
||||
│
|
||||
├── 1. Поиск потомков:
|
||||
│ for i in 0..slots.len():
|
||||
│ if slots[i].parent_idx == Some(slot_idx):
|
||||
│ revoke_internal(i) ← рекурсивно!
|
||||
│
|
||||
├── 2. Уничтожение себя:
|
||||
│ cap = Capability::empty()
|
||||
│ parent_idx = None
|
||||
│ token = old_token_sig
|
||||
│
|
||||
└── 3. Отправка token в очередь:
|
||||
if token != 0 && token != 0xDEAD_BEEF:
|
||||
MMU_REVOCATION_QUEUE.push(token)
|
||||
```
|
||||
|
||||
**Почему нет блокировок при рекурсии?**
|
||||
- На каждом шаге проверка `is_child` захватывает и отпускает блокировку.
|
||||
- Рекурсивный вызов происходит **после** освобождения блокировки.
|
||||
- Это предотвращает взаимоблокировки.
|
||||
|
||||
**Фильтр токенов:**
|
||||
- `token == 0`: пустой/невалидный capability.
|
||||
- `token == 0xDEAD_BEEF`: сырой Untyped (для тестов/отладки).
|
||||
- Эти токены не отправляются в очередь (бессмысленно).
|
||||
|
||||
### get_cap(slot) — чтение
|
||||
|
||||
```rust
|
||||
pub fn get_cap(&self, slot: usize) -> Option<Capability> {
|
||||
self.slots.get(slot).map(|s| s.lock().cap)
|
||||
}
|
||||
```
|
||||
|
||||
## Lock Ranking — детали
|
||||
|
||||
```rust
|
||||
let (src_slot, dest_slot) = if src < dest {
|
||||
_guard_low = self.slots[src].lock(); // меньший → первый
|
||||
_guard_high = self.slots[dest].lock(); // больший → второй
|
||||
} else {
|
||||
_guard_low = self.slots[dest].lock();
|
||||
_guard_high = self.slots[src].lock();
|
||||
};
|
||||
```
|
||||
|
||||
**Причина:** если поток A делает mint(5, 10), а поток B делает mint(10, 5),
|
||||
без lock ranking они могут взаимно заблокироваться:
|
||||
- A: lock(5) → ждёт lock(10)
|
||||
- B: lock(10) → ждёт lock(5)
|
||||
|
||||
С lock ranking:
|
||||
- A: lock(5) → lock(10)
|
||||
- B: lock(5) → ждёт... → дождался → lock(10)
|
||||
|
||||
## Интеграция с событиями
|
||||
|
||||
После `revoke()`, токен попадает в `MMU_REVOCATION_QUEUE` (см.
|
||||
`events.rs`). VMM обрабатывает очередь в `process_pending_revocations()`.
|
||||
|
||||
## Особенности реализации
|
||||
|
||||
1. **Vec<Locked<CNodeSlot>>**: каждый слот — отдельная spinlock-ячейка.
|
||||
Это позволяет параллельно читать разные слоты.
|
||||
2. **`pub slots`**: прямой доступ к слоту возможен (для тестов).
|
||||
3. **panic на overflow**: если очередь отзыва переполнена — паника.
|
||||
Это критический сбой подсистемы ресурсов.
|
||||
115
kernel/docs/capability/descriptors.md
Normal file
115
kernel/docs/capability/descriptors.md
Normal file
@@ -0,0 +1,115 @@
|
||||
# Дескрипторы capability: `descriptor.rs`
|
||||
|
||||
## Назначение
|
||||
|
||||
Файл определяет базовые типы capability-системы:
|
||||
что такое capability, какие бывают объекты, права и отношения.
|
||||
|
||||
## CapObject — что представляет capability
|
||||
|
||||
```rust
|
||||
pub enum CapObject {
|
||||
Empty, // Пустой слот
|
||||
Memory { phys: PhysAddr, size_pages: usize }, // Фрейм памяти
|
||||
CNode { phys: PhysAddr, slots: usize }, // Другой CNode
|
||||
PMActor { id: u64 }, // PM Actor
|
||||
}
|
||||
```
|
||||
|
||||
### Memory
|
||||
|
||||
Capability на физическую память:
|
||||
- `phys` — физический адрес начала.
|
||||
- `size_pages` — размер в страницах.
|
||||
|
||||
### CNode
|
||||
|
||||
Capability на другой CNode:
|
||||
- `phys` — физический адрес CNode.
|
||||
- `slots` — количество слотов.
|
||||
|
||||
### PMActor
|
||||
|
||||
Capability на PMActor:
|
||||
- `id` — уникальный идентификатор актора.
|
||||
|
||||
### Empty
|
||||
|
||||
Слот пуст. `is_valid()` возвращает `false`.
|
||||
|
||||
## CapRights — права доступа
|
||||
|
||||
```rust
|
||||
bitflags! {
|
||||
pub struct CapRights: u8 {
|
||||
const READ = 1 << 0; // 0x01 — чтение
|
||||
const WRITE = 1 << 1; // 0x02 — запись
|
||||
const EXECUTE = 1 << 2; // 0x04 — исполнение
|
||||
const GRANT = 1 << 3; // 0x08 — разрешение на mint
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Правила:**
|
||||
- Права дочернего capability = `родительские_права & запрошенные_права`.
|
||||
- Нельзя расширить права: если родитель не имеет GRANT, mint невозможен.
|
||||
- `CapRights::all()` = R | W | X | G = 0x0F.
|
||||
|
||||
## Relation — тип связи
|
||||
|
||||
```rust
|
||||
pub enum Relation {
|
||||
Strong, // Владелец — сильная ссылка (объект жив, пока есть Strong)
|
||||
Borrow, // Заёмщик — временный доступ
|
||||
Transfer, // Передача — владение переходит без возможности отзыва
|
||||
}
|
||||
```
|
||||
|
||||
- **Strong**: capability владеет объектом. При revoke, объект может
|
||||
быть уничтожен или возвращён пулу.
|
||||
- **Borrow**: capability предоставляет временный доступ.
|
||||
При revoke родителя, borrow-потомки тоже отзываются.
|
||||
- **Transfer**: полная передача владения. Используется при IPC.
|
||||
|
||||
## Capability — полный дескриптор
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct Capability {
|
||||
pub object: CapObject, // Целевой объект
|
||||
pub rights: CapRights, // Права доступа
|
||||
pub relation: Relation, // Тип связи
|
||||
pub token_sig: u64, // Уникальный подписывающий токен
|
||||
}
|
||||
```
|
||||
|
||||
### token_sig — назначение
|
||||
|
||||
- Уникальный 64-битный идентификатор capability.
|
||||
- Используется для:
|
||||
1. **Отзыва**: при revoke, token_sig отправляется в MMU_REVOCATION_QUEUE.
|
||||
2. **Идентификации в VMM**: VMA хранят `cap_token == token_sig`.
|
||||
3. **Отладки**: каждый capability можно однозначно отследить.
|
||||
|
||||
### Методы
|
||||
|
||||
```rust
|
||||
impl Capability {
|
||||
pub const fn empty() -> Self {
|
||||
Self {
|
||||
object: CapObject::Empty,
|
||||
rights: CapRights::empty(),
|
||||
relation: Relation::Borrow,
|
||||
token_sig: 0,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn is_valid(&self) -> bool {
|
||||
!matches!(self.object, CapObject::Empty)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`token_sig = 0` зарезервирован для пустых/невалидных capability.
|
||||
`token_sig = 0xDEAD_BEEF` используется для сырых Untyped
|
||||
(не отправляется в MMU_REVOCATION_QUEUE, см. `cap/mod.rs`).
|
||||
126
kernel/docs/capability/introduction.md
Normal file
126
kernel/docs/capability/introduction.md
Normal file
@@ -0,0 +1,126 @@
|
||||
# Capability-система: концептуальная модель
|
||||
|
||||
## Философия
|
||||
|
||||
**Capability** (дескриптор возможности) — это **неподделываемый токен**,
|
||||
дающий право выполнить определённую операцию над определённым объектом.
|
||||
|
||||
В традиционных ОС (Linux, Windows) доступ контролируется через:
|
||||
- PID + UID/GID + проверка при каждом системном вызове.
|
||||
- MMU: page tables определяют, что отображено, но не кто отобразил.
|
||||
|
||||
В модели capabilities:
|
||||
- **Если у вас нет capability — у вас нет доступа.**
|
||||
- Capability хранятся в CNode — защищённой таблице, доступной только ядру.
|
||||
- Capability можно создавать только от родительского capability
|
||||
(иерархия наследования).
|
||||
- Права можно только **урезать** (mint), но не расширить.
|
||||
- Capability можно **отозвать** (revoke), что уничтожает его
|
||||
и всех его потомков.
|
||||
|
||||
## Основные понятия
|
||||
|
||||
```
|
||||
Capability {
|
||||
object: CapObject, // на что указывает (Memory, CNode, PMActor...)
|
||||
rights: CapRights, // права (R, W, X, G)
|
||||
relation: Relation, // Strong (владеет), Borrow (заём), Transfer
|
||||
token_sig: u64, // уникальный идентификатор (для revoke)
|
||||
}
|
||||
|
||||
Relation {
|
||||
Strong: владеет объектом (capability владеет памятью)
|
||||
Borrow: заём — временный доступ без права распоряжаться
|
||||
Transfer: передача — владение переходит получателю
|
||||
}
|
||||
|
||||
CapRights {
|
||||
READ = 0x1,
|
||||
WRITE = 0x2,
|
||||
EXECUTE = 0x4,
|
||||
GRANT = 0x8, // разрешение создавать дочерние capability
|
||||
}
|
||||
```
|
||||
|
||||
## Иерархия и отзыв
|
||||
|
||||
```
|
||||
CNode (массив слотов)
|
||||
┌────────┬────────┬────────┬────────┐
|
||||
│ slot 0 │ slot 1 │ slot 2 │ slot 3 │ ...
|
||||
├────────┼────────┼────────┼────────┤
|
||||
│ cap │ cap │ cap │ cap │
|
||||
│ parent:│ parent:│ parent:│ parent:│
|
||||
│ None │ Some(0)│ Some(0)│ Some(1)│
|
||||
└────────┴────────┴────────┴────────┘
|
||||
│
|
||||
┌────────┴────────┐
|
||||
▼ ▼
|
||||
slot 1 slot 2
|
||||
(mint from 0) (mint from 0)
|
||||
|
||||
revoke(0) → slot 0 уничтожается
|
||||
→ рекурсивно: slot 1, slot 2 тоже уничтожаются
|
||||
→ token_sig slot 0 отправляется в MMU_REVOCATION_QUEUE
|
||||
→ VMM обработает отзыв при следующем page fault
|
||||
```
|
||||
|
||||
### Mint — создание дочернего capability
|
||||
|
||||
```rust
|
||||
cnode.mint(src, dest, relation, rights)
|
||||
```
|
||||
|
||||
- `src` — исходный слот (должен иметь GRANT).
|
||||
- `dest` — целевой слот (должен быть пустым).
|
||||
- `relation` — как наследник связан с родителем.
|
||||
- `rights` — права наследника (∩ с правами родителя).
|
||||
|
||||
### Revoke — отзыв capability
|
||||
|
||||
```rust
|
||||
cnode.revoke(slot_idx)
|
||||
```
|
||||
|
||||
1. Рекурсивно находит всех потомков и уничтожает их.
|
||||
2. Уничтожает сам capability.
|
||||
3. Отправляет `token_sig` в глобальную `MMU_REVOCATION_QUEUE`.
|
||||
4. VMM при следующем page fault обрабатывает все накопленные отзывы.
|
||||
|
||||
## Lock Ranking — предотвращение дедлоков
|
||||
|
||||
В `mint()` захватываются две блокировки (src и dest). Чтобы избежать
|
||||
инверсии блокировок, используется строгий порядок:
|
||||
|
||||
```rust
|
||||
let (src_slot, dest_slot) = if src < dest {
|
||||
// Захватываем меньший индекс первым
|
||||
_guard_low = self.slots[src].lock();
|
||||
_guard_high = self.slots[dest].lock();
|
||||
} else {
|
||||
_guard_low = self.slots[dest].lock();
|
||||
_guard_high = self.slots[src].lock();
|
||||
};
|
||||
```
|
||||
|
||||
## Интеграция с VMM
|
||||
|
||||
При отзыве capability, VMM должна аннулировать все VMA, связанные
|
||||
с отозванным токеном. Для этого:
|
||||
|
||||
1. `revoke()` пушит `token_sig` в lock-free очередь.
|
||||
2. При page fault: `process_pending_revocations()` дренирует очередь.
|
||||
3. VMA с `cap_token == token_sig` удаляются из AddressSpace.
|
||||
4. TLB flush для синхронизации MMU.
|
||||
|
||||
## Использование в kmain()
|
||||
|
||||
```rust
|
||||
let root_cnode = CNode::new(256);
|
||||
// Вставка capability на фрейм
|
||||
root_cnode.insert(0, mem_cap).unwrap();
|
||||
// Mint с урезанными правами
|
||||
root_cnode.mint(0, 10, Relation::Borrow, R | W).unwrap();
|
||||
// Revoke — отзыв всех потомков
|
||||
root_cnode.revoke(0);
|
||||
```
|
||||
73
kernel/docs/capability/objects.md
Normal file
73
kernel/docs/capability/objects.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# Ядерные объекты: `object.rs`
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
`KernelObject` — это ref-counted представление объекта ядра.
|
||||
В отличие от `Capability` (которая указывает на объект), `KernelObject`
|
||||
— это сам объект с подсчётом ссылок.
|
||||
|
||||
## Структура
|
||||
|
||||
```rust
|
||||
pub struct KernelObject {
|
||||
pub phys_addr: PhysAddr, // физический адрес объекта
|
||||
pub size_bits: u8, // размер в битах (2^size_bits)
|
||||
pub obj_type: ObjectType, // тип объекта
|
||||
pub ref_count: AtomicUsize, // счётчик ссылок
|
||||
pub owner_id: u64, // ID владельца
|
||||
}
|
||||
```
|
||||
|
||||
## ObjectType — классификация объектов
|
||||
|
||||
```rust
|
||||
pub enum ObjectType {
|
||||
Untyped, // Сырая память без типа
|
||||
Frame, // Фрейм (4KiB страница)
|
||||
CNode, // Capability Node
|
||||
ThreadBlock, // Блок управления потоком (TCB)
|
||||
PageTable, // Таблица страниц
|
||||
}
|
||||
```
|
||||
|
||||
## Ref-counting
|
||||
|
||||
```rust
|
||||
impl KernelObject {
|
||||
pub fn add_ref(&self) {
|
||||
self.ref_count.fetch_add(1, Ordering::Relaxed);
|
||||
// ^ Relaxed: нас не волнует порядок других операций
|
||||
// при увеличении счётчика. Только атомарность.
|
||||
}
|
||||
|
||||
pub fn release(&self) -> bool {
|
||||
self.ref_count.fetch_sub(1, Ordering::Release) == 1
|
||||
// ^ Release: все операции до release видны тому,
|
||||
// кто Acquire-читает ref_count.
|
||||
// Возвращает true, если это была последняя ссылка
|
||||
// (объект должен быть уничтожен).
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Текущее состояние
|
||||
|
||||
`KernelObject` определён, но пока не используется активно.
|
||||
Capability-система в текущей версии работает напрямую с `CapObject`
|
||||
и `PhysAddr`, без обёртки в `KernelObject`.
|
||||
|
||||
Планируемое использование:
|
||||
- При выделении памяти через PMActor: создаётся KernelObject.
|
||||
- Capability ссылается на KernelObject через ID/индекс.
|
||||
- Когда последняя Strong capability удалена → KernelObject
|
||||
уничтожается → память возвращается.
|
||||
|
||||
## Отличие CapObject vs KernelObject
|
||||
|
||||
| Характеристика | CapObject | KernelObject |
|
||||
|---------------|-----------|--------------|
|
||||
| Роль | Что capability представляет | Сам объект в памяти ядра |
|
||||
| Ref-count | Нет | AtomicUsize |
|
||||
| Хранение | В CNode slot | В отдельной таблице |
|
||||
| Типы | Memory, CNode, PMActor | Untyped, Frame, CNode, ThreadBlock, PageTable |
|
||||
| Связь | CapObject.Memory.phys = KernelObject.phys_addr | — |
|
||||
115
kernel/docs/cpu/idt.md
Normal file
115
kernel/docs/cpu/idt.md
Normal file
@@ -0,0 +1,115 @@
|
||||
# Interrupt Descriptor Table: `idt.rs`
|
||||
|
||||
## Аппаратная модель
|
||||
|
||||
IDT (Interrupt Descriptor Table) — это таблица из 256 entry (по 16 байт каждая),
|
||||
которая сообщает CPU, куда передавать управление при прерываниях и исключениях.
|
||||
|
||||
```
|
||||
IDT:
|
||||
┌──────┬──────────────────────────────────────────────────────┐
|
||||
│ 0 │ #DE — Divide Error │
|
||||
│ 1 │ #DB — Debug │
|
||||
│ 2 │ #NMI — Non-Maskable Interrupt │
|
||||
│ 3 │ #BP — Breakpoint │
|
||||
│ 4 │ #OF — Overflow │
|
||||
│ 5 │ #BR — Bound Range Exceeded │
|
||||
│ 6 │ #UD — Undefined Opcode │
|
||||
│ 7 │ #NM — Device Not Available │
|
||||
│ 8 │ #DF — Double Fault │
|
||||
│ 9 │ #MF — Coprocessor Segment Overrun │
|
||||
│ 10 │ #TS — Invalid TSS │
|
||||
│ 11 │ #NP — Segment Not Present │
|
||||
│ 12 │ #SS — Stack-Segment Fault │
|
||||
│ 13 │ #GP — General Protection Fault │
|
||||
│ 14 │ #PF — Page Fault │
|
||||
│ 15-31│ Reserved / CPU exceptions │
|
||||
│ 32-255│ User-defined (hardware interrupts) │
|
||||
└──────┴──────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Загрузка IDT: инструкция `lidt [idtr_ptr]`, где `idtr_ptr` — это
|
||||
6-байтовая структура `IdtPtr`:
|
||||
```
|
||||
IdtPtr:
|
||||
┌──────────┬──────────┐
|
||||
│ limit:16 │ base:48 │
|
||||
└──────────┴──────────┘
|
||||
```
|
||||
|
||||
## IdtEntry — 16-байтовая запись
|
||||
|
||||
```rust
|
||||
#[repr(C, packed)]
|
||||
pub struct IdtEntry {
|
||||
offset_low: u16, // Бит 0:15 адреса обработчика
|
||||
selector: u16, // Селектор сегмента кода (0x28 для ядра)
|
||||
ist: u8, // Interrupt Stack Table
|
||||
type_attr: u8, // Тип вентиля + флаги
|
||||
offset_mid: u16, // Бит 16:31 адреса обработчика
|
||||
offset_high: u32, // Бит 32:63 адреса обработчика
|
||||
ignore: u32, // Зарезервировано
|
||||
}
|
||||
```
|
||||
|
||||
**type_attr:**
|
||||
- Бит 7: Present (должен быть 1)
|
||||
- Бит 6-5: DPL (Descriptor Privilege Level)
|
||||
- Бит 4: Reserved (0)
|
||||
- Бит 3-0: Gate Type (0xE = Interrupt Gate, 0xF = Trap Gate)
|
||||
|
||||
`set_handler(handler, selector, flags)`:
|
||||
```rust
|
||||
pub fn set_handler(&mut self, handler: u64, selector: u16, flags: u8) {
|
||||
self.offset_low = handler as u16;
|
||||
self.selector = selector;
|
||||
self.ist = 0;
|
||||
self.type_attr = flags | 0x80; // Present bit forced on
|
||||
self.offset_mid = (handler >> 16) as u16;
|
||||
self.offset_high = (handler >> 32) as u32;
|
||||
}
|
||||
```
|
||||
|
||||
## InterruptDescriptorTable — 256 entry
|
||||
|
||||
```rust
|
||||
pub struct InterruptDescriptorTable {
|
||||
entries: [IdtEntry; 256], // 256 × 16 = 4096 байт
|
||||
}
|
||||
```
|
||||
|
||||
**Методы:**
|
||||
- `set_handler(vector, handler)` — устанавливает обработчик с
|
||||
селектором 0x28 (GDT code segment) и type_attr 0x8E
|
||||
(Interrupt Gate, Ring 0, Present).
|
||||
- `load()` — `lidt` инструкция.
|
||||
|
||||
```rust
|
||||
pub unsafe fn load(&'static self) {
|
||||
let ptr = IdtPtr {
|
||||
limit: (size_of::<Self>() - 1) as u16, // 4095
|
||||
base: self as *const _ as u64,
|
||||
};
|
||||
asm!("lidt [{}]", in(reg) &ptr);
|
||||
}
|
||||
```
|
||||
|
||||
## Глобальная IDT
|
||||
|
||||
```rust
|
||||
pub static mut IDT: InterruptDescriptorTable = InterruptDescriptorTable::new();
|
||||
```
|
||||
|
||||
`static mut` — потому что IDT модифицируется в ранней инициализации,
|
||||
до включения прерываний. Потенциально может быть заменён на `static`
|
||||
с `UnsafeCell`.
|
||||
|
||||
## Детали конфигурации
|
||||
|
||||
- **Selector**: `0x28` — это GDT entry для ring 0 code segment
|
||||
(дескриптор 5, 5 × 8 = 0x28). Селектор сегмента кода в long mode.
|
||||
- **Type 0x8E**: `1000_1110` = бит 7 (Present) + бит 3:0 = 1110
|
||||
(Interrupt Gate, 32-bit). В 64-bit режиме все вентили — 64-bit,
|
||||
флаг 0xE остаётся корректным.
|
||||
- **IST**: 0 — не используем Interrupt Stack Table (один стек для
|
||||
всех обработчиков).
|
||||
200
kernel/docs/cpu/interrupts.md
Normal file
200
kernel/docs/cpu/interrupts.md
Normal file
@@ -0,0 +1,200 @@
|
||||
# Обработчики прерываний и исключений: `interrupts.rs`
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
Файл объединяет **обработчики исключений** (таблицу IDT) и
|
||||
**межпроцессорные прерывания** (TLB shootdown). Это связующий слой
|
||||
между аппаратурой (CPU exceptions, APIC) и программными подсистемами
|
||||
(VMM, Memory).
|
||||
|
||||
## Структура обработчика (stub + Rust handler)
|
||||
|
||||
Каждый обработчик состоит из двух частей:
|
||||
|
||||
1. **Сборочный stub** (global_asm): сохраняет контекст, вызывает
|
||||
Rust-функцию, восстанавливает контекст, iretq.
|
||||
2. **Rust handler**: собственно обработка.
|
||||
|
||||
```
|
||||
Пример: Page Fault
|
||||
[stack]
|
||||
page_fault_stub: error_code ← CPU пушет
|
||||
push rax ...регистры...
|
||||
push rcx
|
||||
...
|
||||
mov rdi, [rsp + 15*8] ← error_code как аргумент
|
||||
call rust_page_fault_handler ← вызов Rust
|
||||
pop r15 восстановление
|
||||
...
|
||||
pop rax
|
||||
add rsp, 8 ← убираем error_code
|
||||
iretq ← возврат
|
||||
```
|
||||
|
||||
## Exception stubs (макрос)
|
||||
|
||||
```rust
|
||||
macro_rules! exception_stub {
|
||||
($name:ident, $handler:ident) => {
|
||||
concat!(
|
||||
".global ", stringify!($name), "\n",
|
||||
stringify!($name), ":\n",
|
||||
"push rax\npush rcx\n...push r15\n", // сохранение
|
||||
"mov rdi, [rsp + 15*8]\n", // error_code
|
||||
"call ", stringify!($handler), "\n",
|
||||
"pop r15\n...pop rax\n", // восстановление
|
||||
"add rsp, 8\n", // очистка error_code
|
||||
"iretq\n",
|
||||
)
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
Используется для:
|
||||
- `page_fault_stub` → `rust_page_fault_handler` (вектор 14)
|
||||
- `gpf_stub` → `rust_gpf_handler` (вектор 13)
|
||||
- `double_fault_stub` → `rust_double_fault_handler` (вектор 8)
|
||||
|
||||
## TLB shootdown stub
|
||||
|
||||
```rust
|
||||
global_asm!(
|
||||
"tlb_shootdown_stub:",
|
||||
"push rax\npush rcx\n...", // сохранение
|
||||
"call rust_tlb_shootdown_handler",
|
||||
"pop r15\n...pop rax\n", // восстановление
|
||||
"iretq"
|
||||
);
|
||||
```
|
||||
|
||||
## Early handlers — для начальной загрузки
|
||||
|
||||
До того как VMM и slab allocator готовы, ядро не может обрабатывать
|
||||
сложные исключения. Для векторов 0-31 генерируются ранние заглушки.
|
||||
|
||||
### Генерация (макрос `.altmacro`)
|
||||
|
||||
```asm
|
||||
.macro early_stub vec
|
||||
.globl early_handler_\vec
|
||||
.balign 16
|
||||
early_handler_\vec:
|
||||
push 0 /* dummy error code (если нет аппаратного) */
|
||||
push \vec /* номер вектора */
|
||||
jmp early_common
|
||||
.endm
|
||||
```
|
||||
|
||||
`early_stub 0`..`early_stub 31` генерирует 32 обработчика.
|
||||
|
||||
### `early_common`
|
||||
|
||||
```asm
|
||||
early_common:
|
||||
push rax\npush rcx\n... // сохранение всех GP-регистров
|
||||
mov rdi, [rsp + 15*8] // vector number
|
||||
mov rsi, [rsp + 16*8] // error code (или dummy 0)
|
||||
call rust_early_exception_handler
|
||||
// never returns
|
||||
```
|
||||
|
||||
### `rust_early_exception_handler(vector, error_code)`
|
||||
|
||||
```rust
|
||||
pub extern "C" fn rust_early_exception_handler(vector: u64, _error_code: u64) -> ! {
|
||||
serial::write_global(format_args!(
|
||||
"\n!!! EARLY EXCEPTION !!! vector={} error_code={:#x}\nCPU halted.\n",
|
||||
vector, _error_code
|
||||
));
|
||||
loop { asm!("cli; hlt"); }
|
||||
}
|
||||
```
|
||||
|
||||
Всегда HALT — раннее исключение фатально.
|
||||
|
||||
## init_early_exceptions() — настройка IDT для ранней загрузки
|
||||
|
||||
```rust
|
||||
pub fn init_early_exceptions() {
|
||||
let idt = addr_of_mut!(IDT);
|
||||
// Устанавливаем early_handler_N для векторов 0-31
|
||||
for (v, &handler) in early_handlers.iter().enumerate() {
|
||||
(*idt).set_handler(v as u8, handler);
|
||||
}
|
||||
// Переопределяем критические:
|
||||
(*idt).set_handler(8, double_fault_stub); // #DF
|
||||
(*idt).set_handler(13, gpf_stub); // #GPF
|
||||
(*idt).set_handler(14, page_fault_stub); // #PF
|
||||
(*idt).set_handler(TLB_SHOOTDOWN_VECTOR, tlb_shootdown_stub);
|
||||
|
||||
let ptr: &'static IDT = &*addr_of!(IDT);
|
||||
ptr.load(); // lidt
|
||||
}
|
||||
```
|
||||
|
||||
## init_idt() — перезагрузка после полной инициализации
|
||||
|
||||
```rust
|
||||
pub fn init_idt() {
|
||||
// Переустанавливаем только Page Fault и TLB Shootdown
|
||||
// (их обработчики уже переключились на VMM-aware версии)
|
||||
let idt = addr_of_mut!(IDT);
|
||||
(*idt).set_handler(14, page_fault_stub);
|
||||
(*idt).set_handler(TLB_SHOOTDOWN_VECTOR, tlb_shootdown_stub);
|
||||
let ptr: &'static IDT = &*addr_of!(IDT);
|
||||
ptr.load();
|
||||
}
|
||||
```
|
||||
|
||||
## Обработчики исключений
|
||||
|
||||
### `rust_page_fault_handler(error_code)`
|
||||
|
||||
```rust
|
||||
pub extern "C" fn rust_page_fault_handler(error_code: u64) {
|
||||
let fault_addr: u64;
|
||||
asm!("mov {}, cr2", out(reg) fault_addr); // читаем CR2
|
||||
|
||||
let write = (error_code & 0x2) != 0; // fault на запись?
|
||||
let present = (error_code & 0x1) != 0; // PTE был PRESENT?
|
||||
|
||||
let mut vmm_guard = KERNEL_SPACE.lock();
|
||||
if let Some(space) = vmm_guard.as_mut() {
|
||||
space.process_pending_revocations(); // обрабатываем отзывы
|
||||
match space.handle_fault(fault_addr, write) {
|
||||
Ok(_) => {} // обработано: iretq retry
|
||||
Err(e) => panic!(...), // необработанный fault
|
||||
}
|
||||
} else {
|
||||
panic!("Page fault before KERNEL_SPACE!");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `rust_gpf_handler(error_code)` и `rust_double_fault_handler(error_code)`
|
||||
|
||||
Оба — HALT с сообщением.
|
||||
|
||||
## TLB Shootdown обработчик
|
||||
|
||||
```rust
|
||||
pub extern "C" fn rust_tlb_shootdown_handler() {
|
||||
crate::mem::vmm::handle_tlb_shootdown_ipi();
|
||||
crate::cpu::lapic::send_eoi(); // подтверждаем LAPIC прерывание
|
||||
}
|
||||
```
|
||||
|
||||
### Тайминги и безопасность
|
||||
|
||||
- TLB shootdown — IPI, требует минимальной задержки.
|
||||
- Все операции в `handle_tlb_shootdown_ipi` — простые и быстрые.
|
||||
- LAPIC EOI отправляется сразу после локального TLB flush.
|
||||
|
||||
## Константы
|
||||
|
||||
```rust
|
||||
pub const TLB_SHOOTDOWN_VECTOR: u8 = 0xFD;
|
||||
```
|
||||
|
||||
Вектор 0xFD (253) — в диапазоне пользовательских прерываний (32-255),
|
||||
намеренно далеко от системных векторов 0-31.
|
||||
106
kernel/docs/cpu/introduction.md
Normal file
106
kernel/docs/cpu/introduction.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# CPU подсистема: концептуальная модель
|
||||
|
||||
## Состав и ответственность
|
||||
|
||||
CPU подсистема отвечает за:
|
||||
|
||||
1. **Обработку прерываний и исключений** — IDT, обработчики.
|
||||
2. **Исключения ранней загрузки** — пока ядро ещё не полностью инициализировано.
|
||||
3. **TLB Shootdown** — межпроцессорное прерывание для синхронизации TLB.
|
||||
4. **Local APIC** — программируемый контроллер прерываний.
|
||||
|
||||
```
|
||||
CPU Subsystem
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ CPU Core #0 │
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌─────────────────────────┐ │
|
||||
│ │ LAPIC │ │ IDT │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ ICR ────────┼────────► 0: #DE (Divide Error) │ │
|
||||
│ │ EOI │ │ 1: #DB (Debug) │ │
|
||||
│ │ (timer) │ │ ... │ │
|
||||
│ │ │ │ 8: #DF (Double Fault) │ │
|
||||
│ │ │ │ 13: #GP (GPF) │ │
|
||||
│ │ │ │ 14: #PF (Page Fault) │ │
|
||||
│ │ │ │ ... │ │
|
||||
│ │ │ │ 0xFD: TLB Shootdown │ │
|
||||
│ └──────────────┘ └─────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Модули
|
||||
|
||||
| Файл | Компонент | Функция |
|
||||
|------|-----------|---------|
|
||||
| `idt.rs` | IDT структуры | Определение IdtEntry, IdtPtr, InterruptDescriptorTable |
|
||||
| `interrupts.rs` | Обработчики | Early handlers, Page Fault, Double Fault, GPF, TLB shootdown |
|
||||
| `lapic.rs` | Local APIC | Инициализация LAPIC, EOI, broadcast IPI |
|
||||
|
||||
## Порядок инициализации
|
||||
|
||||
```
|
||||
kmain()
|
||||
│
|
||||
├── cpu::interrupts::init_early_exceptions()
|
||||
│ └── IDT для векторов 0-31 + TLB shootdown (0xFD)
|
||||
│
|
||||
├── ... (PMM, LAPIC, Page tables) ...
|
||||
│
|
||||
├── cpu::interrupts::init_idt()
|
||||
│ └── Перезагрузка IDT с полными обработчиками
|
||||
│
|
||||
├── STI (разрешение прерываний)
|
||||
│
|
||||
└── ... (работа с прерываниями)
|
||||
```
|
||||
|
||||
## Обработка исключений: два этапа
|
||||
|
||||
### Этап 1: Early (до VMM)
|
||||
|
||||
На раннем этапе загрузки (до настройки page tables и heap) ядро
|
||||
не может обрабатывать сложные исключения. Для векторов 0-31
|
||||
устанавливаются `early_handler_N`, которые:
|
||||
|
||||
1. Пушат вектор и (возможно) dummy error code.
|
||||
2. Переходят в `early_common`.
|
||||
3. Сохраняют все регистры.
|
||||
4. Вызывают `rust_early_exception_handler(vector, error_code)`.
|
||||
5. Паникуют (HALT).
|
||||
|
||||
### Этап 2: Полный (после VMM)
|
||||
|
||||
После инициализации VMM три критических исключения получают
|
||||
полноценные обработчики:
|
||||
|
||||
- **Page Fault (#PF, вектор 14)**: попытка обработать (COW, lazy).
|
||||
- **Double Fault (#DF, вектор 8)**: HALT с сообщением.
|
||||
- **General Protection Fault (#GPF, вектор 13)**: HALT с сообщением.
|
||||
- **TLB Shootdown (вектор 0xFD)**: межпроцессорный TLB сброс.
|
||||
|
||||
## TLB Shootdown: модель
|
||||
|
||||
При изменении page tables на одном ядре, TLB других ядер устаревает.
|
||||
Протокол:
|
||||
|
||||
```
|
||||
CPU 0 (инициатор) CPU 1 (мишень)
|
||||
│ │
|
||||
├── local_tlb_flush_asid() │
|
||||
├── SHOOTDOWN_LOCK.lock() │
|
||||
├── SHOOTDOWN_ASID = asid │
|
||||
├── SHOOTDOWN_ACK = 0 │
|
||||
├── LAPIC: broadcast IPI │
|
||||
│ (вектор TLB_SHOOTDOWN_VECTOR) ───► прерывание!
|
||||
│ ├── handle_tlb_shootdown_ipi()
|
||||
│ ├── local_tlb_flush_asid()
|
||||
│ ├── SHOOTDOWN_ACK |= 1 << core
|
||||
│ ├── LAPIC::send_eoi()
|
||||
│ └── iretq
|
||||
│ │
|
||||
├── spin_loop() ◄────────────────────┤
|
||||
│ (ждёт ACK от всех ядер) │
|
||||
├── SHOOTDOWN_LOCK.unlock() │
|
||||
└── continue │
|
||||
```
|
||||
115
kernel/docs/cpu/lapic.md
Normal file
115
kernel/docs/cpu/lapic.md
Normal file
@@ -0,0 +1,115 @@
|
||||
# Local APIC: `lapic.rs`
|
||||
|
||||
## Аппаратная модель
|
||||
|
||||
**Local APIC** (Advanced Programmable Interrupt Controller) — это
|
||||
встроенный в каждое ядро x86-64 контроллер прерываний.
|
||||
|
||||
```
|
||||
Local APIC (MMIO, начиная с 0xFEE00_000)
|
||||
┌──────────────────────────────┐
|
||||
│ 0x020: IRR (In-Service Reg) │
|
||||
│ 0x030: TMR (Trigger Mode) │
|
||||
│ 0x080: EOI │ ← запись 0 = подтверждение прерывания
|
||||
│ 0x0B0: LINT0/LINT1 │
|
||||
│ 0x0E0: Timer │
|
||||
│ 0x200: LVT Error │
|
||||
│ 0x300: ICR (Interrupt Cmd) │ ← отправка межпроцессорного прерывания
|
||||
│ 0x310: ICR_HIGH (APIC ID) │
|
||||
└──────────────────────────────┘
|
||||
```
|
||||
|
||||
## Регистры
|
||||
|
||||
| Смещение | Регистр | Назначение |
|
||||
|----------|---------|------------|
|
||||
| 0x0B0 | EOI | End Of Interrupt — подтверждение обработки |
|
||||
| 0x300 | ICR (low) | Interrupt Command Register — отправка IPI |
|
||||
| 0x310 | ICR (high) | Destination APIC ID |
|
||||
|
||||
## Инициализация
|
||||
|
||||
```rust
|
||||
static LAPIC_VIRT_BASE: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
pub fn init() {
|
||||
// LAPIC отображён bootloader'ом в HHDM по адресу 0xFEE00_000
|
||||
LAPIC_VIRT_BASE.store(LAPIC_DEFAULT_BASE + get_hhdm(), Ordering::SeqCst);
|
||||
}
|
||||
```
|
||||
|
||||
LAPIC расположен по физическому адресу `0xFEE0_0000`. Ядро получает
|
||||
виртуальный базовый адрес, добавляя HHDM offset.
|
||||
|
||||
## Доступ к регистрам
|
||||
|
||||
```rust
|
||||
#[inline(always)]
|
||||
fn write_lapic_reg(offset: u64, value: u32) {
|
||||
let base = LAPIC_VIRT_BASE.load(Ordering::Relaxed);
|
||||
if base == 0 { return; } // LAPIC ещё не инициализирован
|
||||
unsafe { core::ptr::write_volatile((base + offset) as *mut u32, value) }
|
||||
}
|
||||
```
|
||||
|
||||
- `write_volatile` — запрещает компилятору оптимизировать обращение
|
||||
(регистры MMIO).
|
||||
- `Relaxed` ordering — для отладки/инициализации достаточно.
|
||||
|
||||
## Операции
|
||||
|
||||
### send_eoi() — конец прерывания
|
||||
|
||||
```rust
|
||||
pub fn send_eoi() {
|
||||
write_lapic_reg(LAPIC_EOI, 0); // 0x0B0
|
||||
}
|
||||
```
|
||||
|
||||
### broadcast_ipi_exclude_self(vector) — IPI всем, кроме себя
|
||||
|
||||
```rust
|
||||
pub fn broadcast_ipi_exclude_self(vector: u8) {
|
||||
// ICR[19:18] = 10b (All Excluding Self)
|
||||
// ICR[14] = 1 (Assert)
|
||||
// ICR[7:0] = vector
|
||||
let icr_low = (2 << 18) | (1 << 14) | (vector as u32);
|
||||
write_lapic_reg(LAPIC_ICR_LOW, icr_low);
|
||||
}
|
||||
```
|
||||
|
||||
Используется для TLB shootdown — необходимо разослать всем ядрам
|
||||
(кроме текущего) IPI с вектором `TLB_SHOOTDOWN_VECTOR`.
|
||||
|
||||
### current_core_id() — определение текущего ядра
|
||||
|
||||
```rust
|
||||
pub fn current_core_id() -> u32 {
|
||||
// CPUID leaf 1: EBX[31:24] = Local APIC ID
|
||||
let mut ebx: u32;
|
||||
asm!(
|
||||
"mov {tmp:r}, rbx", // спрятать rbx (резерв LLVM)
|
||||
"mov eax, 1", "cpuid",
|
||||
"mov {out:e}, ebx", // сохранить EBX
|
||||
"mov rbx, {tmp:r}", // восстановить rbx
|
||||
tmp = out(reg) _,
|
||||
out = out(reg) ebx,
|
||||
...
|
||||
);
|
||||
ebx >> 24
|
||||
}
|
||||
```
|
||||
|
||||
**RBX проблема:** LLVM резервирует RBX, поэтому его нужно сохранять
|
||||
и восстанавливать вручную вокруг CPUID инструкции.
|
||||
|
||||
## Тонкости
|
||||
|
||||
1. **MMIO vs MSR**: LAPIC можно программировать через MSR (IA32_APIC_BASE)
|
||||
и через MMIO. Bootloader (Limine) настраивает MMIO mapping в HHDM.
|
||||
2. **Инициализация**: LAPIC уже включён bootloader'ом. `init()` просто
|
||||
сохраняет виртуальный адрес.
|
||||
3. **x2APIC**: не используется (в текущей версии — MMIO xAPIC).
|
||||
4. **EOI**: обязателен после каждого прерывания от LAPIC (включая IPI).
|
||||
5. **ICR запись**: после записи в ICR Low, шина APIC доставляет
|
||||
прерывание. Запись блокирующая (ждёт готовности шины).
|
||||
88
kernel/docs/debug/introduction.md
Normal file
88
kernel/docs/debug/introduction.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# Подсистема отладки: концептуальная модель
|
||||
|
||||
## Два канала вывода
|
||||
|
||||
Ядро имеет два параллельных канала для отладки:
|
||||
|
||||
1. **Экранный (framebuffer console)** — через `tty::Console`.
|
||||
- Использует PSF2-шрифты.
|
||||
- Цветной вывод (зелёный/жёлтый/красный для Info/Warn/Error).
|
||||
- Медленнее, но визуально нагляднее.
|
||||
|
||||
2. **Serial port (COM1)** — через `debug::serial`.
|
||||
- Текстовый вывод с ANSI escape codes.
|
||||
- Работает через QEMU/KVM serial console.
|
||||
- Быстрее, может быть перенаправлен в файл.
|
||||
|
||||
## LogLevel — уровни логирования
|
||||
|
||||
```rust
|
||||
pub enum LogLevel {
|
||||
Info, // Зелёный на экране, зелёный в serial
|
||||
Warn, // Жёлтый
|
||||
Error, // Красный
|
||||
}
|
||||
```
|
||||
|
||||
Каждый уровень имеет:
|
||||
- `serial_color_code()` — ANSI escape code для serial.
|
||||
- `console_color()` — RGB значение для framebuffer.
|
||||
|
||||
## Макросы
|
||||
|
||||
```rust
|
||||
// Основной макрос
|
||||
log!(console, level, module, format_args...)
|
||||
|
||||
// Специализированные
|
||||
info!(console, module, format_args...)
|
||||
warn!(console, module, format_args...)
|
||||
error!(console, module, format_args...)
|
||||
```
|
||||
|
||||
**Формат вывода на экран:**
|
||||
```
|
||||
[ LOG ] <module> | <message>
|
||||
```
|
||||
|
||||
**Формат в serial:**
|
||||
```
|
||||
GREEN[ LOG] RESET <module> | <message>
|
||||
```
|
||||
|
||||
## Цветовое кодирование
|
||||
|
||||
| Уровень | Экран (RGB) | Serial (ANSI) |
|
||||
|---------|-------------|---------------|
|
||||
| Info | 0x00FF00 | `\x1b[32m` (green) |
|
||||
| Warn | 0xFFFF00 | `\x1b[33m` (yellow) |
|
||||
| Error | 0xFF0000 | `\x1b[31m` (red) |
|
||||
| Текст | 0xFFFFFF (white) | `\x1b[0m` (reset) |
|
||||
|
||||
## Использование в kmain()
|
||||
|
||||
```rust
|
||||
// После инициализации serial
|
||||
debug::serial::init_global();
|
||||
|
||||
// После инициализации console
|
||||
info!(console, "BOOT", "LIS4 Kernel Starting...");
|
||||
info!(console, "MEM", "BitmapPMM initialized.");
|
||||
info!(console, "LAPIC", "Local APIC initialized.");
|
||||
```
|
||||
|
||||
## Архитектура
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌───────────────────┐
|
||||
│ kmain() │────►│ log!() macro │
|
||||
└──────────────┘ └────────┬──────────┘
|
||||
│
|
||||
┌──────────────┼──────────────┐
|
||||
▼ ▼ ▼
|
||||
┌──────────┐ ┌──────────┐ ┌──────────┐
|
||||
│ Экран │ │ Serial │ │ Паника │
|
||||
│ Console │ │ COM1 │ │ Handler │
|
||||
│ (tty.rs) │ │(serial.rs)│ │(main.rs) │
|
||||
└──────────┘ └──────────┘ └──────────┘
|
||||
```
|
||||
110
kernel/docs/debug/serial.md
Normal file
110
kernel/docs/debug/serial.md
Normal file
@@ -0,0 +1,110 @@
|
||||
# Serial Port драйвер: `serial.rs`
|
||||
|
||||
## Назначение
|
||||
|
||||
Драйвер последовательного порта (UART 16550, COM1) для отладочного
|
||||
вывода. Позволяет видеть сообщения ядра через QEMU serial console,
|
||||
minicom, screen и т.д.
|
||||
|
||||
## Аппаратная модель
|
||||
|
||||
COM1 расположен по портам ввода-вывода `0x3F8`-`0x3FF`:
|
||||
|
||||
| Порт | Регистр | Назначение |
|
||||
|------|---------|------------|
|
||||
| 0x3F8 | DATA | Чтение/запись данных |
|
||||
| 0x3F9 | IER | Interrupt Enable |
|
||||
| 0x3FA | IIR/FCR | Interrupt ID / FIFO Control |
|
||||
| 0x3FB | LCR | Line Control |
|
||||
| 0x3FC | MCR | Modem Control |
|
||||
| 0x3FD | LSR | Line Status |
|
||||
| 0x3FE | MSR | Modem Status |
|
||||
|
||||
## Инициализация
|
||||
|
||||
```rust
|
||||
pub unsafe fn init() -> Self {
|
||||
let port = Self::COM1; // 0x3F8
|
||||
outb(port + 1, 0x00); // IER = 0 (disable interrupts)
|
||||
outb(port + 3, 0x80); // LCR DLAB=1 (enable baud rate programming)
|
||||
outb(port + 0, 0x03); // Divisor LSB = 3 (38400 baud)
|
||||
outb(port + 1, 0x00); // Divisor MSB = 0
|
||||
outb(port + 3, 0x03); // LCR = 8N1 (8 bits, No parity, 1 stop)
|
||||
outb(port + 2, 0xC7); // FCR = enable FIFO, clear, 14-byte threshold
|
||||
outb(port + 4, 0x0B); // MCR = DTR+RTS+OUT2 (enable IRQ + handshake)
|
||||
SerialPort(port)
|
||||
}
|
||||
```
|
||||
|
||||
### Детали конфигурации
|
||||
|
||||
1. **IER = 0**: отключаем прерывания UART (TODO: включить для RX).
|
||||
2. **DLAB = 1**: разрешаем программирование делителя бода.
|
||||
3. **Divisor = 3**: при тактовой 1.8432 MHz → 115200 / 3 = 38400 бод.
|
||||
4. **LCR = 0x03**: 8N1 — 8 бит данных, нет чётности, 1 стоп-бит.
|
||||
5. **FCR = 0xC7**: enable FIFO, clear both FIFOs, trigger at 14 bytes.
|
||||
6. **MCR = 0x0B**: DTR=1, RTS=1, OUT2=1 (необходимо для IRQ на ISA шине).
|
||||
|
||||
## Отправка байта
|
||||
|
||||
```rust
|
||||
fn is_transmit_empty(&self) -> bool {
|
||||
unsafe { (inb(self.0 + 5) & 0x20) != 0 } // LSR bit 5 = Transmitter Holding Register Empty
|
||||
}
|
||||
|
||||
pub fn send(&self, data: u8) {
|
||||
while !self.is_transmit_empty() {} // Ждём, пока UART готов
|
||||
unsafe { outb(self.0, data); }
|
||||
}
|
||||
```
|
||||
|
||||
## fmt::Write реализация
|
||||
|
||||
```rust
|
||||
impl core::fmt::Write for SerialPort {
|
||||
fn write_str(&mut self, s: &str) -> core::fmt::Result {
|
||||
for byte in s.bytes() { self.send(byte); }
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Глобальный экземпляр
|
||||
|
||||
```rust
|
||||
static SERIAL_PORT: Locked<Option<SerialPort>> = Locked::new(None);
|
||||
|
||||
pub fn init_global() {
|
||||
let mut guard = SERIAL_PORT.lock();
|
||||
*guard = Some(unsafe { SerialPort::init() });
|
||||
}
|
||||
|
||||
pub fn write_global(args: core::fmt::Arguments) {
|
||||
let mut guard = SERIAL_PORT.lock();
|
||||
if let Some(ref mut sp) = *guard {
|
||||
let _ = sp.write_fmt(args);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`init_global()` вызывается из `kmain()` после инициализации TTY.
|
||||
`write_global()` используется:
|
||||
- В `log!()` макросе (через debug.rs).
|
||||
- В `rust_panic()` обработчике.
|
||||
- В обработчиках исключений (GPF, Double Fault, Early Exception).
|
||||
|
||||
## Низкоуровневый I/O
|
||||
|
||||
```rust
|
||||
unsafe fn outb(port: u16, val: u8) {
|
||||
asm!("out dx, al", in("dx") port, in("al") val,
|
||||
options(nomem, nostack, preserves_flags));
|
||||
}
|
||||
|
||||
unsafe fn inb(port: u16) -> u8 {
|
||||
let res: u8;
|
||||
asm!("in al, dx", out("al") res, in("dx") port,
|
||||
options(nomem, nostack, preserves_flags));
|
||||
res
|
||||
}
|
||||
```
|
||||
128
kernel/docs/events/introduction.md
Normal file
128
kernel/docs/events/introduction.md
Normal file
@@ -0,0 +1,128 @@
|
||||
# Очередь отзыва RevocationQueue: `events.rs`
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
**RevocationQueue** — это глобальная lock-free кольцевая очередь,
|
||||
которая соединяет capability-систему (CNode) с виртуальной памятью (VMM).
|
||||
|
||||
### Назначение
|
||||
|
||||
Когда capability отзывается, VMM должна аннулировать все VMA,
|
||||
связанные с этим capability. Но:
|
||||
- CNode.revoke() не может напрямую обратиться к VMM (разные подсистемы,
|
||||
разное время жизни блокировок).
|
||||
- VMM может быть в обработке page fault'а.
|
||||
- Отзыв может быть каскадным и рекурсивным.
|
||||
|
||||
Решение: **асинхронная очередь**.
|
||||
|
||||
```
|
||||
CNode.revoke(token_1)
|
||||
│
|
||||
└──→ push(token_1) ───→ MMU_REVOCATION_QUEUE ──→ pop() ──→ VMM
|
||||
│
|
||||
CNode.revoke(token_2) │
|
||||
│ │
|
||||
└──→ push(token_2) ─────────────────────────────────────────┤
|
||||
│
|
||||
Page Fault handler │
|
||||
│ │
|
||||
└──→ process_pending_revocations() ←────────────────────────┘
|
||||
```
|
||||
|
||||
## RevocationQueue — детали
|
||||
|
||||
```rust
|
||||
pub struct RevocationQueue {
|
||||
buffer: [AtomicU64; QUEUE_SIZE], // 1024 entry
|
||||
_pad0: [u8; 64], // padding: кэш-линия
|
||||
head: AtomicUsize, // читатель (VMM)
|
||||
_pad1: [u8; 64], // padding: кэш-линия
|
||||
tail: AtomicUsize, // писатели (CNode revoke)
|
||||
}
|
||||
```
|
||||
|
||||
Тот же паттерн, что и в `PMActorQueue`: разделение head и tail
|
||||
по разным кэш-линиям предотвращает false sharing.
|
||||
|
||||
### push(token_sig)
|
||||
|
||||
```rust
|
||||
pub fn push(&self, token_sig: u64) -> Result<(), &'static str> {
|
||||
let mut tail = self.tail.load(Ordering::Relaxed);
|
||||
loop {
|
||||
let head = self.head.load(Ordering::Acquire);
|
||||
if tail.wrapping_sub(head) >= QUEUE_SIZE {
|
||||
return Err("Revocation queue overflow");
|
||||
}
|
||||
match self.tail.compare_exchange_weak(tail, tail+1, AcqRel, Relaxed) {
|
||||
Ok(_) => {
|
||||
self.buffer[tail & QUEUE_MASK].store(token_sig, Release);
|
||||
return Ok(());
|
||||
}
|
||||
Err(actual) => tail = actual,
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### pop()
|
||||
|
||||
```rust
|
||||
pub fn pop(&self) -> Option<u64> {
|
||||
let head = self.head.load(Ordering::Relaxed);
|
||||
let tail = self.tail.load(Ordering::Acquire);
|
||||
if head == tail { return None; }
|
||||
let token = self.buffer[head & QUEUE_MASK].load(Ordering::Acquire);
|
||||
self.head.store(head.wrapping_add(1), Ordering::Release);
|
||||
Some(token)
|
||||
}
|
||||
```
|
||||
|
||||
## Глобальный экземпляр
|
||||
|
||||
```rust
|
||||
pub static MMU_REVOCATION_QUEUE: RevocationQueue = RevocationQueue::new();
|
||||
```
|
||||
|
||||
## VMM: обработка отзывов
|
||||
|
||||
```rust
|
||||
// В interrupts.rs: rust_page_fault_handler()
|
||||
let mut vmm_guard = KERNEL_SPACE.lock();
|
||||
if let Some(space) = vmm_guard.as_mut() {
|
||||
space.process_pending_revocations(); // ← дренирует очередь
|
||||
match space.handle_fault(fault_addr, write) { ... }
|
||||
}
|
||||
```
|
||||
|
||||
```rust
|
||||
// В vmm.rs: AddressSpace
|
||||
pub fn process_pending_revocations(&mut self) {
|
||||
let mut needs_flush = false;
|
||||
while let Some(token) = MMU_REVOCATION_QUEUE.pop() {
|
||||
self.do_revoke_by_token(token);
|
||||
needs_flush = true;
|
||||
}
|
||||
if needs_flush {
|
||||
tlb_flush_asid(self.asid);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Почему именно так?
|
||||
|
||||
1. **Lock-free**: отзыв capability не блокируется VMM.
|
||||
2. **Batch processing**: VMM обрабатывает сразу все накопленные отзывы
|
||||
при следующем page fault, экономя TLB flush'ы.
|
||||
3. **Deadlock prevention**: process_pending_revocations вызывается
|
||||
**до** handle_fault, избегая циклических зависимостей между
|
||||
блокировками capability и VMM.
|
||||
4. **Простота**: кольцевой буфер с CAS — проверенная временем структура.
|
||||
|
||||
## Ограничения
|
||||
|
||||
- Размер: 1024 entry. Если очередь переполняется — **паника**.
|
||||
Это проектный выбор: переполнение означает, что VMM не успевает
|
||||
обрабатывать отзывы быстрее, чем CNode генерирует новые.
|
||||
- Однопоточный pop (только page fault handler дренирует очередь).
|
||||
1055
kernel/docs/glossary/terms.md
Normal file
1055
kernel/docs/glossary/terms.md
Normal file
File diff suppressed because it is too large
Load Diff
70
kernel/docs/index.md
Normal file
70
kernel/docs/index.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# Документация ядра Elyz (LISA)
|
||||
|
||||
## Структура документации
|
||||
|
||||
```
|
||||
kernel/docs/
|
||||
├── index.md ← этот файл: оглавление
|
||||
│
|
||||
├── overview/
|
||||
│ ├── introduction.md ← Концептуальная модель (7 принципов)
|
||||
│ └── architecture.md ← Архитектура, схема, порядок инициализации
|
||||
│
|
||||
├── memory/
|
||||
│ ├── introduction.md ← Трёхуровневая модель памяти
|
||||
│ ├── address-translation.md ← PhysAddr, VirtAddr, HHDM
|
||||
│ ├── physical-memory.md ← BitmapPMM (глобальный)
|
||||
│ ├── paging.md ← PageTable, 4-level walk
|
||||
│ ├── buddy.md ← BuddyAllocator (intrusive list)
|
||||
│ ├── pm-actor.md ← PMActor + MPSC очередь
|
||||
│ ├── pm-router.md ← PMRouter (65536 каналов)
|
||||
│ ├── allocator.md ← Slab-аллокатор (куча)
|
||||
│ └── vmm.md ← AddressSpace, VMA, COW, TLB
|
||||
│
|
||||
├── cpu/
|
||||
│ ├── introduction.md ← CPU подсистема, 2 этапа обработки
|
||||
│ ├── idt.md ← IDT entry, InterruptDescriptorTable
|
||||
│ ├── interrupts.md ← Обработчики, stubs, early exceptions
|
||||
│ └── lapic.md ← Local APIC, IPI, EOI
|
||||
│
|
||||
├── capability/
|
||||
│ ├── introduction.md ← Capability model, mint, revoke
|
||||
│ ├── descriptors.md ← Capability, CapObject, CapRights
|
||||
│ ├── cnode.md ← CNode, lock ranking
|
||||
│ └── objects.md ← KernelObject (ref-counted)
|
||||
│
|
||||
├── debug/
|
||||
│ ├── introduction.md ← Отладка: экран + serial
|
||||
│ └── serial.md ← COM1 UART 16550 драйвер
|
||||
│
|
||||
├── boot/
|
||||
│ ├── introduction.md ← Процесс загрузки, Limine
|
||||
│ └── linker.md ← Линкер-скрипты
|
||||
│
|
||||
└── events/
|
||||
└── introduction.md ← RevocationQueue (lock-free)
|
||||
```
|
||||
|
||||
## Быстрый старт по файлам
|
||||
|
||||
| Файл ядра | Что делает | Документация |
|
||||
|-----------|-----------|--------------|
|
||||
| `src/main.rs` | Точка входа, инициализация, тесты | `overview/architecture.md` (раздел порядка инициализации) |
|
||||
| `src/mem/pmm.rs` | Глобальный PMM (Bitmap + L1) | `memory/physical-memory.md` |
|
||||
| `src/mem/buddy.rs` | Buddy-аллокатор | `memory/buddy.md` |
|
||||
| `src/mem/paging.rs` | 4-уровневые page tables | `memory/paging.md` |
|
||||
| `src/mem/pm_manages.rs` | PMActor + очередь | `memory/pm-actor.md` |
|
||||
| `src/mem/pm_router.rs` | PM Router каналы | `memory/pm-router.md` |
|
||||
| `src/mem/vmm.rs` | AddressSpace, COW, TLB | `memory/vmm.md` |
|
||||
| `src/mem/allocator.rs` | Slab-аллокатор | `memory/allocator.md` |
|
||||
| `src/mem/address.rs` | Физические/виртуальные адреса | `memory/address-translation.md` |
|
||||
| `src/cpu/idt.rs` | IDT структуры | `cpu/idt.md` |
|
||||
| `src/cpu/interrupts.rs` | Обработчики исключений | `cpu/interrupts.md` |
|
||||
| `src/cpu/lapic.rs` | Local APIC | `cpu/lapic.md` |
|
||||
| `src/cap/mod.rs` | CNode | `capability/cnode.md` |
|
||||
| `src/cap/descriptor.rs` | Capability типы | `capability/descriptors.md` |
|
||||
| `src/cap/object.rs` | KernelObject | `capability/objects.md` |
|
||||
| `src/debug/mod.rs` | Log macros | `debug/introduction.md` |
|
||||
| `src/debug/serial.rs` | Serial driver | `debug/serial.md` |
|
||||
| `src/events.rs` | Очередь отзыва | `events/introduction.md` |
|
||||
| `src/tty.rs` | Framebuffer console | `tty.md` |
|
||||
103
kernel/docs/memory/address-translation.md
Normal file
103
kernel/docs/memory/address-translation.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# Адресация и трансляция: `address.rs`
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
В x86-64 MMU транслирует виртуальные адреса (48 бит, канонические)
|
||||
в физические (до 52 бит) через 4-уровневую иерархию page tables.
|
||||
|
||||
Ядро использует **Higher-Half Direct Map (HHDM)** — область виртуальной
|
||||
памяти, где вся физическая память отображена 1:1 с фиксированным смещением.
|
||||
|
||||
```
|
||||
Физический адрес 0x0
|
||||
│
|
||||
│ + HHDM_OFFSET (например, 0xFFFF8000_0000_0000)
|
||||
▼
|
||||
Виртуальный адрес 0xFFFF8000_0000_0000
|
||||
```
|
||||
|
||||
## Физический адрес: `PhysAddr(pub u64)`
|
||||
|
||||
Представляет собой физический адрес. Это **newtype** над `u64` — строгая
|
||||
типизация предотвращает случайное смешивание физических и виртуальных адресов.
|
||||
|
||||
**Методы:**
|
||||
- `to_virt()` — преобразует в виртуальный адрес через HHDM:
|
||||
`VirtAddr(self.0 + HHDM_OFFSET)`
|
||||
- `is_aligned()` — проверка выравнивания на 4KiB
|
||||
- `align_down()` / `align_up()` — выравнивание вниз/вверх до границы 4KiB
|
||||
|
||||
## Виртуальный адрес: `VirtAddr(pub u64)`
|
||||
|
||||
**Методы:**
|
||||
- `to_phys()` — обратное преобразование: вычитает HHDM, возвращает `None`
|
||||
если адрес ниже HHDM (не канонический для физической памяти)
|
||||
- `as_ptr<T>()` — получить `*const T` (для чтения)
|
||||
- `as_mut_ptr<T>()` — получить `*mut T` (для записи)
|
||||
|
||||
## HHDM Offset: глобальное состояние
|
||||
|
||||
```rust
|
||||
static HHDM_OFFSET: AtomicU64 = AtomicU64::new(0);
|
||||
```
|
||||
|
||||
- `init_hhdm(offset: u64)` — вызывается в `kmain()` сразу после получения
|
||||
ответа от bootloader'а. Сохраняет offset через `Ordering::Release`.
|
||||
- `get_hhdm() -> u64` — читает offset через `Ordering::Relaxed` (высокая
|
||||
производительность, на этой архитектуре атомарность гарантирована).
|
||||
|
||||
### Почему Atomic?
|
||||
|
||||
HHDM инициализируется однократно до включения прерываний и создания
|
||||
дополнительных потоков. Atomic гарантирует, что даже если в будущем
|
||||
инструкции будут переупорядочены, значение будет видно всем ядрам.
|
||||
|
||||
## Код и абстракция
|
||||
|
||||
```rust
|
||||
// address.rs (полный код)
|
||||
static HHDM_OFFSET: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
pub fn init_hhdm(offset: u64) {
|
||||
HHDM_OFFSET.store(offset, Ordering::Release);
|
||||
// ^ Release: все предыдущие записи становятся видимы для
|
||||
// всех acquire-операций на HHDM_OFFSET
|
||||
}
|
||||
|
||||
pub fn get_hhdm() -> u64 {
|
||||
HHDM_OFFSET.load(Ordering::Relaxed)
|
||||
// ^ Relaxed: в однопоточном контексте и при инициализации
|
||||
// до включения прерываний это безопасно и максимально быстро
|
||||
}
|
||||
|
||||
#[repr(transparent)]
|
||||
pub struct PhysAddr(pub u64);
|
||||
// ^ repr(transparent): гарантирует, что PhysAddr имеет то же
|
||||
// представление в памяти, что и u64 — важно для FFI и передачи
|
||||
// в ассемблерные инструкции (mov cr3, rax)
|
||||
|
||||
impl PhysAddr {
|
||||
pub fn to_virt(self) -> VirtAddr {
|
||||
VirtAddr(self.0 + get_hhdm())
|
||||
// ^ Концептуально: физический адрес + HHDM смещение даёт
|
||||
// виртуальный адрес, по которому этот физический фрейм
|
||||
// отображён в higher half
|
||||
}
|
||||
}
|
||||
|
||||
impl VirtAddr {
|
||||
pub fn as_mut_ptr<T>(self) -> *mut T {
|
||||
self.0 as *mut T
|
||||
// ^ Прямая интерпретация числа как указателя.
|
||||
// Безопасность на стороне вызывающего кода.
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Где используется
|
||||
|
||||
- `PhysAddr::to_virt()` — в `paging.rs` для доступа к Page Table entries,
|
||||
в `pmm.rs` для обнуления фреймов, в `buddy.rs` для intrusive list.
|
||||
- `VirtAddr::as_mut_ptr()` — в `paging.rs` для создания ссылок на PTE.
|
||||
- `init_hhdm()` — в `kmain()` при старте.
|
||||
- `get_hhdm()` — в `lapic.rs` для вычисления LAPIC_base.
|
||||
211
kernel/docs/memory/allocator.md
Normal file
211
kernel/docs/memory/allocator.md
Normal file
@@ -0,0 +1,211 @@
|
||||
# Slab-аллокатор: куча ядра
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
**SlabAllocator** — это глобальный аллокатор кучи для ядра.
|
||||
Rust-программы используют `alloc::vec::Vec`, `alloc::boxed::Box` и т.д. —
|
||||
все они в конечном счёте вызывают `GlobalAlloc::alloc()`.
|
||||
|
||||
### Стратегия
|
||||
|
||||
Для маленьких блоков (≤ 2048 байт) — **slab lists**:
|
||||
предварительно нарезанные блоки фиксированного размера.
|
||||
|
||||
Для больших блоков (> 2048 байт) — **freelist больших блоков**:
|
||||
освобождённые блоки переиспользуются.
|
||||
|
||||
Если ни там, ни там нет — **bump allocation**:
|
||||
последовательная раздача из заранее выделенного региона.
|
||||
|
||||
```
|
||||
Запрос alloc(32 байта):
|
||||
1. list_index(32) = 2 (BLOCK_SIZES[2] = 32)
|
||||
2. list_heads[2] есть свободный блок?
|
||||
- Да: отдаём его, заменяем голову списка
|
||||
- Нет: bump-аллокация блока размером 32 (fallback_alloc)
|
||||
|
||||
Запрос alloc(4096 байт):
|
||||
1. list_index(4096) = None (максимум 2048)
|
||||
2. large_block_free есть блок ≥ 4096?
|
||||
- Да: отдаём
|
||||
- Нет: bump-аллокация
|
||||
```
|
||||
|
||||
## Структуры данных
|
||||
|
||||
### Slab списки — `list_heads`
|
||||
|
||||
```rust
|
||||
const BLOCK_SIZES: &[usize] = &[8, 16, 32, 64, 128, 256, 512, 1024, 2048];
|
||||
|
||||
struct ListNode {
|
||||
next: Option<&'static mut ListNode>,
|
||||
}
|
||||
|
||||
pub struct SlabAllocator {
|
||||
list_heads: [Option<&'static mut ListNode>; BLOCK_SIZES.len()], // 9 списков
|
||||
large_block_free: Option<&'static mut LargeBlockNode>,
|
||||
heap_start: usize,
|
||||
heap_end: usize,
|
||||
next_bump: usize,
|
||||
}
|
||||
```
|
||||
|
||||
### LargeBlockNode — для блоков > 2048 байт
|
||||
|
||||
```rust
|
||||
struct LargeBlockNode {
|
||||
size: usize,
|
||||
next: Option<&'static mut LargeBlockNode>,
|
||||
}
|
||||
```
|
||||
|
||||
### Инициализация: `init(start, size)`
|
||||
|
||||
```rust
|
||||
pub fn init(&mut self, start: usize, size: usize) {
|
||||
self.heap_start = start;
|
||||
self.next_bump = start;
|
||||
self.heap_end = start + size;
|
||||
}
|
||||
```
|
||||
|
||||
Вызывается в kmain() после настройки page table для области `0xFFFF_9000_0000_0000`.
|
||||
|
||||
## GlobalAlloc — реализация
|
||||
|
||||
### alloc(layout)
|
||||
|
||||
```rust
|
||||
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
|
||||
let mut allocator = self.lock();
|
||||
match SlabAllocator::list_index(&layout) {
|
||||
Some(index) => {
|
||||
// 1. Пробуем slab list
|
||||
match allocator.list_heads[index].take() {
|
||||
Some(node) => {
|
||||
allocator.list_heads[index] = node.next.take();
|
||||
node as *mut ListNode as *mut u8
|
||||
}
|
||||
None => {
|
||||
// 2. Нет в slab — bump alloc целого блока
|
||||
let block_size = BLOCK_SIZES[index];
|
||||
allocator.fallback_alloc(
|
||||
Layout::from_size_align(block_size, block_size).unwrap()
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
None => allocator.fallback_alloc(layout) // > 2048
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### fallback_alloc(layout)
|
||||
|
||||
```rust
|
||||
fn fallback_alloc(&mut self, layout: Layout) -> *mut u8 {
|
||||
let size = layout.size().max(layout.align());
|
||||
|
||||
// 1. Пробуем large block free list (для > 2048)
|
||||
if size > 2048 {
|
||||
// поиск по large_block_free
|
||||
while let Some(ref mut node) = *field {
|
||||
if node.size >= size {
|
||||
// отдаём, удаляем из списка
|
||||
return node as *mut u8;
|
||||
}
|
||||
field = &mut node.next;
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Bump alloc
|
||||
let alloc_start = (self.next_bump + layout.align() - 1) & !(layout.align() - 1);
|
||||
let alloc_end = alloc_start.checked_add(layout.size())?;
|
||||
if alloc_end > self.heap_end {
|
||||
null_mut() // OOM
|
||||
} else {
|
||||
self.next_bump = alloc_end;
|
||||
alloc_start as *mut u8
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### dealloc(ptr, layout)
|
||||
|
||||
```rust
|
||||
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
|
||||
let mut allocator = self.lock();
|
||||
match SlabAllocator::list_index(&layout) {
|
||||
Some(index) => {
|
||||
// Добавляем в slab list (переиспользование)
|
||||
let new_node = ListNode { next: allocator.list_heads[index].take() };
|
||||
let new_node_ptr = ptr as *mut ListNode;
|
||||
unsafe { new_node_ptr.write(new_node); }
|
||||
allocator.list_heads[index] = Some(&mut *new_node_ptr);
|
||||
}
|
||||
None => {
|
||||
// Добавляем в large block free list
|
||||
let new_node = LargeBlockNode {
|
||||
size: layout.size().max(layout.align()),
|
||||
next: allocator.large_block_free.take(),
|
||||
};
|
||||
let new_node_ptr = ptr as *mut LargeBlockNode;
|
||||
unsafe { new_node_ptr.write(new_node); }
|
||||
allocator.large_block_free = Some(&mut *new_node_ptr);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Важно:** `layout.size()` должен быть ≥ `size_of::<ListNode>()`, чтобы
|
||||
освобождённый блок мог хранить указатели списка. Гарантируется, потому
|
||||
что наименьший BLOCK_SIZE (8) ≥ `size_of::<Option<&'static mut ListNode>>`
|
||||
(8 байт на 64-bit).
|
||||
|
||||
## Locked<T> — примитивная spinlock-обёртка
|
||||
|
||||
```rust
|
||||
pub struct Locked<A> {
|
||||
inner: UnsafeCell<A>,
|
||||
lock: AtomicBool,
|
||||
}
|
||||
```
|
||||
|
||||
- `lock()` — spin-wait с CAS + `hint::spin_loop()`.
|
||||
- `LockedGuard` — RAII guard, при Drop отпускает блокировку.
|
||||
- `unsafe impl Sync` — потому что `lock()` гарантирует взаимное исключение.
|
||||
|
||||
Используется не только для аллокатора, но и для PMM, serial port,
|
||||
VMM KERNEL_SPACE и других глобальных структур.
|
||||
|
||||
## Глобальный аллокатор
|
||||
|
||||
```rust
|
||||
#[global_allocator]
|
||||
pub static ALLOCATOR: Locked<SlabAllocator> = Locked::new(SlabAllocator::new());
|
||||
```
|
||||
|
||||
Этот `static` перехватывает все вызовы `alloc::alloc::alloc()`,
|
||||
делая возможным использование `Vec`, `Box`, `String` и т.д. в ядре.
|
||||
|
||||
## Heap init в kmain()
|
||||
|
||||
```rust
|
||||
let heap_start = 0xFFFF_9000_0000_0000;
|
||||
let heap_size = 8 * 1024 * 1024; // 8 MiB
|
||||
// Предварительно map'им все страницы кучи
|
||||
for i in (0..heap_size).step_by(4096) {
|
||||
let frame = mem::pmm::alloc_frame().expect("OOM");
|
||||
p4.map_page(VirtAddr(heap_start + i), frame, flags);
|
||||
}
|
||||
// Инициализируем аллокатор
|
||||
allocator::ALLOCATOR.lock().init(heap_start as usize, heap_size);
|
||||
```
|
||||
|
||||
## Почему slab?
|
||||
|
||||
1. **Скорость**: alloc/dealloc — O(1) для малых блоков.
|
||||
2. **Нет фрагментации**: блоки фиксированного размера.
|
||||
3. **Локальность**: блоки одного размера рядом в памяти.
|
||||
4. **Простота**: ~180 строк кода.
|
||||
267
kernel/docs/memory/buddy.md
Normal file
267
kernel/docs/memory/buddy.md
Normal file
@@ -0,0 +1,267 @@
|
||||
# Buddy Allocator: `buddy.rs`
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
Buddy-аллокатор — это алгоритм управления памятью, который:
|
||||
- Делит память на блоки размером `2^order` страниц.
|
||||
- Каждый блок может быть либо свободен, либо занят.
|
||||
- При освобождении блок объединяется (coalesce) с соседом (buddy),
|
||||
если тот тоже свободен, образуя блок вдвое большего размера.
|
||||
|
||||
```
|
||||
Пример: порядок 0 (1 страница), порядок 1 (2 страницы), порядок 2 (4 страницы)
|
||||
|
||||
Order 2: [ 0-3 ] [ 4-7 ] [ 8-11 ]
|
||||
Order 1: [ 0-1 ][ 2-3 ] [ 4-5 ][ 6-7 ] [ 8-9 ][ 10-11 ]
|
||||
Order 0: [0][1][2][3] [4][5][6][7] [8][9][10][11] [12][13][14][15]
|
||||
^
|
||||
buddy-пара: (0,1), (2,3), (4,5)...
|
||||
buddy(i) = i XOR (1 << order)
|
||||
```
|
||||
|
||||
## Intrusive List (список в самих страницах)
|
||||
|
||||
Вместо отдельной структуры данных для списков свободных блоков,
|
||||
Elyz использует **intrusive linked list** — указатели хранятся прямо
|
||||
внутри свободных физических страниц:
|
||||
|
||||
```
|
||||
Страница (4 KiB):
|
||||
┌──────────────┐
|
||||
│ next: usize │ ← указатель на следующую свободную страницу
|
||||
├──────────────┤
|
||||
│ prev: usize │ ← указатель на предыдущую свободную страницу
|
||||
├──────────────┤
|
||||
│ │
|
||||
│ (не занято) │
|
||||
│ │
|
||||
└──────────────┘
|
||||
```
|
||||
|
||||
Для доступа к странице по индексу используется HHDM:
|
||||
```rust
|
||||
fn page_virt(&self, idx: usize) -> VirtAddr {
|
||||
PhysAddr(self.base_phys + idx as u64 * 4096).to_virt()
|
||||
}
|
||||
```
|
||||
|
||||
## Структура BuddyAllocator
|
||||
|
||||
```rust
|
||||
pub struct BuddyAllocator {
|
||||
free_heads: [usize; MAX_ORDER + 1], // головы списков для каждого порядка
|
||||
total_pages: usize, // всего страниц в управлении
|
||||
free_pages: usize, // свободно страниц
|
||||
base_phys: u64, // физический адрес начала
|
||||
}
|
||||
```
|
||||
|
||||
- `MAX_ORDER = 11` — максимальный порядок (2^11 = 2048 страниц = 8 MiB).
|
||||
- `NEXT_SENTINEL = usize::MAX` — маркер конца списка.
|
||||
- `free_heads[order]` — индекс первой свободной страницы порядка `order`.
|
||||
|
||||
## Операции со списком
|
||||
|
||||
### flist_push(order, idx) — добавить в голову
|
||||
|
||||
```rust
|
||||
fn flist_push(&mut self, order: usize, idx: usize) {
|
||||
let head = self.free_heads[order];
|
||||
// Устанавливаем: node.next = head, node.prev = SENTINEL
|
||||
self.write_node(idx, head, NEXT_SENTINEL);
|
||||
if head != NEXT_SENTINEL {
|
||||
// head.prev = idx
|
||||
self.write_node(head, self.read_next(head), idx);
|
||||
}
|
||||
self.free_heads[order] = idx;
|
||||
}
|
||||
```
|
||||
|
||||
### flist_remove(order, idx) — удалить из списка
|
||||
|
||||
```rust
|
||||
fn flist_remove(&mut self, order: usize, idx: usize) {
|
||||
let next = self.read_next(idx);
|
||||
let prev = self.read_prev(idx);
|
||||
// Очищаем указатели удаляемого узла
|
||||
self.write_node(idx, NEXT_SENTINEL, NEXT_SENTINEL);
|
||||
if prev != NEXT_SENTINEL {
|
||||
self.write_node(prev, next, self.read_prev(prev));
|
||||
} else {
|
||||
self.free_heads[order] = next; // удалили голову
|
||||
}
|
||||
if next != NEXT_SENTINEL {
|
||||
self.write_node(next, self.read_next(next), prev);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### flist_pop(order) — извлечь из головы
|
||||
|
||||
```rust
|
||||
fn flist_pop(&mut self, order: usize) -> Option<usize> {
|
||||
let head = self.free_heads[order];
|
||||
if head == NEXT_SENTINEL { return None; }
|
||||
self.flist_remove(order, head);
|
||||
Some(head)
|
||||
}
|
||||
```
|
||||
|
||||
## Инициализация: `new(total_pages, base_phys)`
|
||||
|
||||
Стратегия: разбить весь диапазон на максимально возможные блоки.
|
||||
|
||||
```rust
|
||||
pub fn new(total_pages: usize, base_phys: u64) -> Self {
|
||||
let mut this = Self {
|
||||
free_heads: [NEXT_SENTINEL; MAX_ORDER + 1],
|
||||
total_pages, free_pages: 0, base_phys,
|
||||
};
|
||||
if total_pages == 0 { return this; }
|
||||
|
||||
let mut idx = 0;
|
||||
while idx < total_pages {
|
||||
let remaining = total_pages - idx;
|
||||
// Максимальный порядок с учётом выравнивания и остатка
|
||||
let align_order = /* макс порядок по выравниванию idx */;
|
||||
let size_order = /* макс порядок по remaining */;
|
||||
let order = MAX_ORDER.min(align_order).min(size_order);
|
||||
let block_size = 1usize << order;
|
||||
this.flist_push(order, idx);
|
||||
this.free_pages += block_size;
|
||||
idx += block_size;
|
||||
}
|
||||
this
|
||||
}
|
||||
```
|
||||
|
||||
### Как определяются align_order и size_order
|
||||
|
||||
- `align_order`: если `idx == 0` — `MAX_ORDER`; иначе `(idx.trailing_zeros()).min(MAX_ORDER)`.
|
||||
Чем больше нулевых битов в idx, тем выше порядок выравнивания.
|
||||
- `size_order`: `(64 - 1) - remaining.leading_zeros()` — максимальная степень двойки
|
||||
≤ remaining.
|
||||
|
||||
## Аллокация: `alloc(order)`
|
||||
|
||||
```rust
|
||||
pub fn alloc(&mut self, order: usize) -> Option<usize> {
|
||||
// 1. Ищем первый непустой список начиная с order
|
||||
let found_order = (order..=MAX_ORDER)
|
||||
.find(|&o| self.free_heads[o] != NEXT_SENTINEL)?;
|
||||
|
||||
// 2. Извлекаем блок из found_order
|
||||
let block_idx = self.flist_pop(found_order);
|
||||
self.free_pages -= 1 << found_order;
|
||||
|
||||
// 3. Разбиваем до нужного порядка (split)
|
||||
let mut cur_order = found_order;
|
||||
while cur_order > order {
|
||||
cur_order -= 1;
|
||||
let buddy_idx = block_idx + (1 << cur_order);
|
||||
self.flist_push(cur_order, buddy_idx);
|
||||
self.free_pages += 1 << cur_order;
|
||||
}
|
||||
|
||||
Some(block_idx)
|
||||
// Возвращается индекс первого блока
|
||||
}
|
||||
```
|
||||
|
||||
### Пример split
|
||||
|
||||
```
|
||||
Запрос: order 1 (2 страницы)
|
||||
Найден: order 3 (8 страниц, блок [0-7])
|
||||
|
||||
Шаг 1: cur_order = 3 → 2
|
||||
buddy = 0 + 4 = 4
|
||||
push(order=2, idx=4) — блок [4-7] в order 2
|
||||
Шаг 2: cur_order = 2 → 1
|
||||
buddy = 0 + 2 = 2
|
||||
push(order=1, idx=2) — блок [2-3] в order 1
|
||||
Результат: order=1, idx=0 — блок [0-1]
|
||||
```
|
||||
|
||||
## Освобождение: `free(block_idx, order)` + coalesce
|
||||
|
||||
```rust
|
||||
pub fn free(&mut self, mut block_idx: usize, mut order: usize) {
|
||||
// Пытаемся объединить с buddy
|
||||
while order < MAX_ORDER {
|
||||
let buddy_idx = block_idx ^ (1 << order);
|
||||
|
||||
// Проверка: buddy в пределах памяти?
|
||||
let buddy_end = buddy_idx.checked_add(1 << order)?;
|
||||
if buddy_end > self.total_pages { break; }
|
||||
|
||||
// Buddy свободен?
|
||||
if self.flist_contains(order, buddy_idx) {
|
||||
// Удаляем buddy из его списка
|
||||
self.flist_remove(order, buddy_idx);
|
||||
self.free_pages -= 1 << order;
|
||||
// Объединяем: block_idx = min(block_idx, buddy_idx)
|
||||
block_idx = block_idx.min(buddy_idx);
|
||||
order += 1;
|
||||
} else {
|
||||
break; // buddy занят, не можем объединить
|
||||
}
|
||||
}
|
||||
|
||||
// Добавляем объединённый блок в список
|
||||
self.free_pages += 1 << order;
|
||||
self.flist_push(order, block_idx);
|
||||
}
|
||||
```
|
||||
|
||||
### Пример coalesce
|
||||
|
||||
```
|
||||
Освобождаем: order 0, idx = 0
|
||||
Блок [0] возвращается в список order 0.
|
||||
|
||||
Проверка: buddy(0, order 0) = 0 XOR 1 = 1
|
||||
Если idx=1 тоже свободен в order 0:
|
||||
- Удаляем idx=1 из order 0
|
||||
- block_idx = min(0, 1) = 0
|
||||
- order = 1
|
||||
- Проверка: buddy(0, order 1) = 0 XOR 2 = 2
|
||||
Если idx=2 свободен в order 1:
|
||||
- Удаляем idx=2 из order 1
|
||||
- order = 2
|
||||
- ...
|
||||
```
|
||||
|
||||
## alloc_pages(page_count) — универсальная аллокация
|
||||
|
||||
```rust
|
||||
pub fn alloc_pages(&mut self, page_count: usize) -> Option<(usize, usize)> {
|
||||
let order = order_for(page_count);
|
||||
self.alloc(order).map(|idx| (idx, order))
|
||||
}
|
||||
```
|
||||
|
||||
`order_for()` вычисляет `ceil(log2(page_count))`.
|
||||
|
||||
## Вспомогательные функции
|
||||
|
||||
- `free_pages()` / `total_pages()` — геттеры.
|
||||
- `is_exhausted()` — `free_pages == 0`.
|
||||
- `stats()` — возвращает `[(order, count)]` для каждого порядка (количество
|
||||
свободных блоков). Используется для отладки.
|
||||
- `flist_contains(order, idx)` — проверка, есть ли блок в списке
|
||||
(проверяет `prev != NEXT_SENTINEL` или `head == idx`).
|
||||
|
||||
## Почему Intrusive List?
|
||||
|
||||
1. **Zero overhead**: не нужно отдельной аллокации для списков.
|
||||
2. **O(1)**: push/pop/remove — все O(1) операции.
|
||||
3. **Константная память**: аллокатор не потребляет дополнительной памяти
|
||||
сверх управляемых страниц.
|
||||
4. **Кэш-дружественность**: доступ к next/prev обращается к той же странице,
|
||||
которая будет выделена.
|
||||
|
||||
## Где используется
|
||||
|
||||
- `PMActor` содержит `BuddyAllocator` для управления своим диапазоном.
|
||||
- Прямые тесты в `kmain()` демонстрируют all/alloc/free/coalesce/exhaustion.
|
||||
82
kernel/docs/memory/introduction.md
Normal file
82
kernel/docs/memory/introduction.md
Normal file
@@ -0,0 +1,82 @@
|
||||
# Управление памятью: концептуальная модель
|
||||
|
||||
## Архитектурная философия
|
||||
|
||||
Управление памятью в Elyz построено как **трёхуровневая иерархия**,
|
||||
где каждый уровень решает свою задачу и взаимодействует с соседними
|
||||
через строго определённые интерфейсы.
|
||||
|
||||
```
|
||||
Уровень 1: BitmapPMM (глобальный, физический)
|
||||
│
|
||||
│ предоставляет сырые фреймы
|
||||
▼
|
||||
Уровень 2: PMActor + BuddyAllocator (распределённый, физический)
|
||||
│
|
||||
│ управляет диапазонами, выдаёт под-диапазоны
|
||||
▼
|
||||
Уровень 3: AddressSpace (VMM) (виртуальный)
|
||||
│
|
||||
│ отображает физические фреймы в виртуальные адреса
|
||||
▼
|
||||
CPU (MMU, page tables)
|
||||
```
|
||||
|
||||
### Зачем три уровня?
|
||||
|
||||
- **BitmapPMM** — глобальный аллокатор физических фреймов. Простой,
|
||||
надёжный, но неэффективный для частых alloc/free маленьких блоков.
|
||||
Используется для начальной загрузки и для Page Table страниц.
|
||||
|
||||
- **BuddyAllocator + PMActor** — распределённая модель. Каждый актор
|
||||
управляет своим диапазоном физической памяти через buddy-алгоритм.
|
||||
Это даёт: (1) изоляцию — актор A не может истощить память актора B;
|
||||
(2) масштабирование — акторы могут работать параллельно;
|
||||
(3) предсказуемость — каждый актор знает свой лимит.
|
||||
|
||||
- **AddressSpace (VMM)** — виртуальные адресные пространства. PML4,
|
||||
VMA-деревья, copy-on-write, lazy mapping. Использует PMM для
|
||||
аллокации Page Table страниц (через `pmm_alloc` в `paging.rs`).
|
||||
|
||||
### Разделение ответственности по файлам
|
||||
|
||||
| Файл | Компонент | Роль в абстракции |
|
||||
|------|-----------|-------------------|
|
||||
| `address.rs` | PhysAddr / VirtAddr / HHDM | Базовые типы для адресов |
|
||||
| `pmm.rs` | BitmapPMM | Глобальный менеджер физических фреймов |
|
||||
| `buddy.rs` | BuddyAllocator | O(1) buddy allocator (intrusive list) |
|
||||
| `pm_manages.rs` | PMActor + PMActorQueue | Актёр физической памяти + MPSC очередь |
|
||||
| `pm_router.rs` | PMRouter | Lock-free маршрутизация запросов/ответов |
|
||||
| `paging.rs` | PageTable | Аппаратные 4-уровневые page tables |
|
||||
| `vmm.rs` | AddressSpace + VMA | Виртуальные адресные пространства |
|
||||
| `allocator.rs` | SlabAllocator | Кучевой аллокатор (global_allocator) |
|
||||
| `mod.rs` | Экспорт | Фасад подсистемы |
|
||||
|
||||
### Поток данных: типичная аллокация
|
||||
|
||||
```
|
||||
Процесс A хочет 16 страниц:
|
||||
1. Код пользователя отправляет PMRequest::Allocate в PMActor через
|
||||
PMRouter::alloc_channel() + actor.submit_request()
|
||||
2. Когда актор получает CPU, process_messages() вызывает
|
||||
buddy.alloc_pages(16)
|
||||
3. BuddyAllocator находит блок порядка 4 (2^4 = 16) или больше,
|
||||
разбивает его при необходимости
|
||||
4. PMActor создаёт Capability с CapObject::Memory { phys, size_pages }
|
||||
5. Ответ (PMResponse::Allocated) отправляется через PMRouter
|
||||
6. Получатель может отобразить фреймы в своё AddressSpace через
|
||||
map_region() или map_shared()
|
||||
```
|
||||
|
||||
### Поток данных: page fault
|
||||
|
||||
```
|
||||
CPU ловит #PF (page fault):
|
||||
1. interrupt.rs: rust_page_fault_handler() читает CR2
|
||||
2. Получает блокировку KERNEL_SPACE
|
||||
3. process_pending_revocations() — обрабатывает накопленные отзывы
|
||||
4. handle_fault() проверяет: это COW? это lazy region?
|
||||
5. Если COW — копируем страницу (copy-on-write)
|
||||
6. Если lazy — alloc_frame() из PMM + map_page()
|
||||
7. Если нераспознанный fault — KERNEL PANIC
|
||||
```
|
||||
196
kernel/docs/memory/paging.md
Normal file
196
kernel/docs/memory/paging.md
Normal file
@@ -0,0 +1,196 @@
|
||||
# Page Tables и отображение памяти: `paging.rs`
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
x86-64 использует **4-уровневую иерархию таблиц страниц**:
|
||||
|
||||
```
|
||||
Виртуальный адрес (48 бит):
|
||||
┌─────────┬─────────┬─────────┬─────────┬─────────┐
|
||||
│ PML4 │ PDPT │ PD │ PT │ offset │
|
||||
│ bits │ bits │ bits │ bits │ (12) │
|
||||
│ 47:39 │ 38:30 │ 29:21 │ 20:12 │ 11:0 │
|
||||
└────┬────┴────┬────┴────┬────┴────┬────┴─────────┘
|
||||
│ │ │ │
|
||||
▼ ▼ ▼ ▼
|
||||
PML4 ──► PDPT ──► PD ──► PT ──► Физический фрейм
|
||||
(512 ent) (512 ent) (512 ent) (512 ent) (4 KiB)
|
||||
```
|
||||
|
||||
Каждая запись (PTE) = 8 байт:
|
||||
```
|
||||
Bit 63: NX (No Execute)
|
||||
Bits 62:52: Ignored/AVL
|
||||
Bits 51:12: Physical frame address (4KiB aligned)
|
||||
Bits 11:9: Ignored/AVL
|
||||
Bits 8: Global
|
||||
Bits 7: PS (Page Size = 1 GiB для PDPT, 2 MiB для PD)
|
||||
Bits 6: Dirty
|
||||
Bits 5: Accessed
|
||||
Bits 4: PCD (Cache Disable)
|
||||
Bits 3: PWT (Write Through)
|
||||
Bits 2: U/S (User/Supervisor)
|
||||
Bits 1: R/W (Read/Write)
|
||||
Bit 0: Present
|
||||
```
|
||||
|
||||
## PageTable — представление таблицы
|
||||
|
||||
```rust
|
||||
#[repr(C, align(4096))]
|
||||
pub struct PageTable {
|
||||
entries: [u64; 512],
|
||||
}
|
||||
```
|
||||
|
||||
- `512` entries × 8 байт = 4096 байт (ровно одна страница).
|
||||
- `align(4096)` — гарантирует, что структура размещается на физической
|
||||
странице, как требует аппаратура.
|
||||
|
||||
## PageTableFlags — битовые флаги
|
||||
|
||||
```rust
|
||||
bitflags! {
|
||||
pub struct PageTableFlags: u64 {
|
||||
const PRESENT = 1 << 0;
|
||||
const WRITABLE = 1 << 1;
|
||||
const USER = 1 << 2;
|
||||
const WRITE_THROUGH = 1 << 3;
|
||||
const NO_CACHE = 1 << 4;
|
||||
const ACCESSED = 1 << 5;
|
||||
const DIRTY = 1 << 6;
|
||||
const HUGE_PAGE = 1 << 7;
|
||||
const GLOBAL = 1 << 8;
|
||||
const COW = 1 << 9; // Elyz-specific: Copy-on-Write marker
|
||||
const NO_EXECUTE = 1 << 63;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Флаг `COW` (бит 9) — Elyz-специфичный, используется в AddressSpace для
|
||||
маркировки страниц, которые нужно копировать при записи.
|
||||
|
||||
## Основные операции PageTable
|
||||
|
||||
### map_page(virt, phys, flags) — отображение одной страницы
|
||||
|
||||
```rust
|
||||
pub fn map_page(&mut self, virt: VirtAddr, phys: PhysAddr, flags: PageTableFlags) {
|
||||
let p1 = self.walk_to_p1_mut(virt, true).expect("...");
|
||||
let p1_idx = ((virt.0 >> 12) & 0x1FF) as usize;
|
||||
p1.entries[p1_idx] = phys.0 | flags.bits();
|
||||
asm!("invlpg [{}]", in(reg) virt.0);
|
||||
}
|
||||
```
|
||||
|
||||
**Процесс:**
|
||||
1. `walk_to_p1_mut(virt, create=true)` проходит 4 уровня (P4→P3→P2→P1),
|
||||
создавая недостающие таблицы через `pmm_alloc()`.
|
||||
2. Записывает физический адрес с флагами в P1 entry.
|
||||
3. `INVLPG` — сбрасывает TLB для этой страницы.
|
||||
|
||||
### map_region(virt, phys, size, flags) — отображение диапазона
|
||||
|
||||
Вызывает `map_page()` для каждой страницы в диапазоне.
|
||||
`size.div_ceil(4096)` — количество страниц.
|
||||
|
||||
### unmap_page(virt) — удаление отображения
|
||||
|
||||
```rust
|
||||
pub fn unmap_page(&mut self, virt: VirtAddr) {
|
||||
// walk_to_p1_mut(virt, false) — не создаём, только ищем
|
||||
let Some(p1) = self.walk_to_p1_mut(virt, false) else { return; };
|
||||
p1.entries[p1_idx] = 0; // Clear PTE
|
||||
asm!("invlpg [{}]", in(reg) virt.0);
|
||||
}
|
||||
```
|
||||
|
||||
**Важно:** не освобождает физический фрейм — это ответственность
|
||||
Caller'а (VMM). Идемпотентна: если PTE нет, ничего не делает.
|
||||
|
||||
### translate(virt) — трансляция виртуального адреса
|
||||
|
||||
Проходит 4 уровня, проверяя PRESENT на каждом уровне.
|
||||
Поддерживает huge pages (1 GiB и 2 MiB):
|
||||
- Если P3 entry имеет HUGE_PAGE — 1 GiB страница.
|
||||
- Если P2 entry имеет HUGE_PAGE — 2 MiB страница.
|
||||
|
||||
### get_flags(virt) / update_flags(virt) — чтение/запись флагов
|
||||
|
||||
`get_flags` возвращает флаги PTE для виртуального адреса.
|
||||
|
||||
`update_flags` меняет флаги, сохраняя физический адрес, и вызывает INVLPG.
|
||||
|
||||
### activate(phys_addr) — загрузка в CR3
|
||||
|
||||
```rust
|
||||
pub unsafe fn activate(&self, phys_addr: PhysAddr) {
|
||||
asm!("mov cr3, {0}", in(reg) phys_addr.0);
|
||||
}
|
||||
```
|
||||
|
||||
Полная замена таблиц страниц. Полный TLB flush (без PCID).
|
||||
|
||||
В `vmm.rs::AddressSpace::activate()` используется улучшенная версия
|
||||
с PCID и битом NOFLUSH.
|
||||
|
||||
## walk_to_p1_mut — сердце навигации
|
||||
|
||||
```rust
|
||||
fn walk_to_p1_mut(&mut self, virt: VirtAddr, create: bool) -> Option<&mut Self> {
|
||||
let p4_idx = ((virt.0 >> 39) & 0x1FF) as usize;
|
||||
let p3_idx = ((virt.0 >> 30) & 0x1FF) as usize;
|
||||
let p2_idx = ((virt.0 >> 21) & 0x1FF) as usize;
|
||||
|
||||
let p3 = self.get_or_create_next_table(p4_idx, create)?;
|
||||
let p2 = p3.get_or_create_next_table(p3_idx, create)?;
|
||||
p2.get_or_create_next_table(p2_idx, create)
|
||||
}
|
||||
```
|
||||
|
||||
### get_or_create_next_table(index, create) — следующий уровень
|
||||
|
||||
```rust
|
||||
fn get_or_create_next_table(&mut self, index: usize, create: bool) -> Option<&mut Self> {
|
||||
if entry & PRESENT == 0 {
|
||||
if !create { return None; }
|
||||
// Аллоцируем новый фрейм для таблицы
|
||||
let pt_phys = pmm_alloc().expect("OOM");
|
||||
// Обнуляем (все entry = 0 = not present)
|
||||
core::ptr::write_bytes(pt_virt.as_mut_ptr::<u8>(), 0, 4096);
|
||||
// Устанавливаем entry: PRESENT | WRITABLE | USER
|
||||
self.entries[index] = pt_phys.0 | (PRESENT | WRITABLE | USER);
|
||||
}
|
||||
// Преобразуем физический адрес entry в ссылку на PageTable
|
||||
let next_phys = PhysAddr(self.entries[index] & PTE_ADDR_MASK);
|
||||
Some(unsafe { &mut *next_phys.to_virt().as_mut_ptr::<Self>() })
|
||||
}
|
||||
```
|
||||
|
||||
### pte_addr_mask — маска адреса
|
||||
|
||||
```rust
|
||||
const PTE_ADDR_MASK: u64 = 0x000F_FFFF_FFFF_F000;
|
||||
```
|
||||
|
||||
Биты 51:12 — физический адрес с выравниванием 4KiB.
|
||||
(0x000F_FFFF_FFFF_F000 = 52 бита адреса, очищенные нижние 12 бит).
|
||||
|
||||
## pmm_alloc — шлюз к PMM
|
||||
|
||||
```rust
|
||||
pub fn pmm_alloc() -> Option<PhysAddr> {
|
||||
PMM.lock().as_mut()?.alloc_frame()
|
||||
}
|
||||
```
|
||||
|
||||
Тонкая обёртка, чтобы избежать циклической зависимости между paging и pmm.
|
||||
|
||||
## Ключевые инварианты
|
||||
|
||||
1. Каждая PageTable занимает ровно одну физическую страницу (4 KiB).
|
||||
2. Физический адрес таблицы всегда 4KiB-выровнен.
|
||||
3. Все intermediate таблицы создаются с флагом USER (чтобы и ядро,
|
||||
и пользователь могли их использовать). Разделение на user/kernel
|
||||
контролируется на уровне P1 entry.
|
||||
4. `INVLPG` вызывается после каждого изменения PTE для консистентности TLB.
|
||||
132
kernel/docs/memory/physical-memory.md
Normal file
132
kernel/docs/memory/physical-memory.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# Глобальный менеджер физической памяти: `pmm.rs`
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
`BitmapPMM` — это **глобальный, единственный в системе**, распределитель
|
||||
физических фреймов (4 KiB). Он отвечает на вопрос:
|
||||
|
||||
> Какой физический фрейм сейчас свободен?
|
||||
|
||||
### Структуры данных
|
||||
|
||||
PMM использует **трёхуровневую битовую карту**:
|
||||
|
||||
```
|
||||
Уровень 1 (L1 bitmap): u64 слова
|
||||
бит в L1 = 1 если соответствующее слово L0 не полностью занято
|
||||
|
||||
Уровень 0 (L0 bitmap): байты
|
||||
каждый байт = 8 фреймов, каждый бит = 1 фрейм
|
||||
бит = 1 → фрейм занят
|
||||
бит = 0 → фрейм свободен
|
||||
|
||||
Ref-counts (счётчики ссылок): u16
|
||||
для каждого фрейма — сколько раз он был захвачен
|
||||
```
|
||||
|
||||
### Зачем L1 (ускорение)?
|
||||
|
||||
Наивный линейный поиск по битовой карте размером в мегабайты — O(N).
|
||||
L1-битмап позволяет пропускать целые группы по 64 слова (4096 фреймов),
|
||||
где нет свободных страниц.
|
||||
|
||||
**Поиск:**
|
||||
1. Ищем `l1_bitmap[l1_idx]` с ненулевым значением (есть свободные).
|
||||
2. Вычисляем `word_idx` по позиции бита внутри L1-слова.
|
||||
3. Читаем `bitmap[word_idx]` как u64.
|
||||
4. Если слово == `!0` (все заняты) — очищаем бит в L1, продолжаем.
|
||||
5. Иначе — `(!word).trailing_zeros()` даёт номер свободного бита.
|
||||
6. Вычисляем `page_idx` и возвращаем `PhysAddr`.
|
||||
|
||||
### Инициализация
|
||||
|
||||
```rust
|
||||
pub unsafe fn init(mmap: &limine::response::MemoryMapResponse) {
|
||||
```
|
||||
|
||||
1. Находим максимальный физический адрес из всех entry карты памяти.
|
||||
2. Вычисляем `total_pages = max_addr / PAGE_SIZE`.
|
||||
3. Вычисляем размер метаданных:
|
||||
- `bitmap_size = total_pages / 8` (1 бит на фрейм)
|
||||
- `ref_counts_size = total_pages * 2` (u16 на фрейм)
|
||||
- `l1_size = (total_pages / 64 / 64) * 8`
|
||||
4. Ищем **первый USABLE регион** размером >= метаданные.
|
||||
5. Размещаем метаданные в этом регионе: сначала bitmap, потом ref_counts,
|
||||
потом L1.
|
||||
6. **Заполняем bitmap = 0xFF** (все занято).
|
||||
7. **Проходим по карте памяти**: для каждого USABLE entry очищаем биты
|
||||
(free_frame).
|
||||
8. **Блокируем фреймы метаданных** (они теперь заняты PMM).
|
||||
9. **Блокируем фрейм 0** (традиционно reserved).
|
||||
10. Сохраняем `PMM.lock() = Some(pmm)`.
|
||||
|
||||
### alloc_frame() — пошагово
|
||||
|
||||
```rust
|
||||
pub fn alloc_frame(&mut self) -> Option<PhysAddr> {
|
||||
```
|
||||
|
||||
1. Двухпроходный поиск: сначала от `last_word` до конца, потом от начала.
|
||||
(`last_word` — хинт для локализации, даёт амортизированное O(1)).
|
||||
2. Для каждого L1-слова: если != 0 — есть свободные.
|
||||
3. `trailing_zeros()` даёт первое слово с хотя бы одним свободным битом.
|
||||
4. Читаем слово как u64.
|
||||
5. `(!word).trailing_zeros()` даёт свободный бит.
|
||||
6. Вычисляем `page_idx`.
|
||||
7. Устанавливаем бит = 1, ref_count = 1, used_pages += 1.
|
||||
8. Если слово стало полностью занятым — очищаем бит в L1.
|
||||
9. Обновляем `last_word`.
|
||||
|
||||
### free_frame() — пошагово
|
||||
|
||||
```rust
|
||||
pub fn free_frame(&mut self, phys_addr: PhysAddr) {
|
||||
```
|
||||
|
||||
1. Если бит уже 1 (фрейм занят):
|
||||
- Уменьшаем ref_count (saturating_sub).
|
||||
- Если ref_count == 0: очищаем бит, used_pages -= 1, обновляем L1.
|
||||
2. Если бит == 0 (уже свободен) — ничего не делаем (идемпотентность).
|
||||
|
||||
### lock_frame() — захват без освобождения
|
||||
|
||||
Используется для резервирования фреймов, которые не должны быть
|
||||
освобождены (нулевой фрейм, фреймы метаданных PMM).
|
||||
|
||||
### inc_ref_frame() — увеличение счётчика
|
||||
|
||||
Используется COW (Copy-on-Write) при fork для учёта разделяемых страниц.
|
||||
|
||||
## Thread Safety
|
||||
|
||||
```rust
|
||||
pub static PMM: Locked<Option<BitmapPMM>> = Locked::new(None);
|
||||
```
|
||||
|
||||
Locked<T> — примитивная spinlock-обёртка (см. `allocator.rs`).
|
||||
Все операции PMM требуют захвата блокировки.
|
||||
|
||||
## Глобальные функции
|
||||
|
||||
В `pmm.rs` определены функции-обёртки для удобства:
|
||||
|
||||
| Функция | Вызывает |
|
||||
|---------|----------|
|
||||
| `alloc_frame()` | `PMM.lock().as_mut()?.alloc_frame()` |
|
||||
| `alloc_contiguous(n)` | `PMM.lock().as_mut()?.alloc_contiguous(n)` |
|
||||
| `free_frame(addr)` | `PMM.lock().as_mut()?.free_frame(addr)` |
|
||||
| `inc_ref_frame(addr)` | `PMM.lock().as_mut()?.inc_ref_frame(addr)` |
|
||||
| `get_stats()` | `PMM.lock().as_ref()?.used_pages/total_pages` |
|
||||
|
||||
## alloc_contiguous() — выделение непрерывной области
|
||||
|
||||
Линейный проход по всем страницам, поиск `count` последовательных
|
||||
свободных фреймов. O(total_pages) — используется редко, только для
|
||||
устройств без IOMMU.
|
||||
|
||||
## Ключевая инварианта
|
||||
|
||||
`BitmapPMM` не знает о buddy-аллокаторах и PMActors. Он просто
|
||||
отвечает на запросы «дай фрейм» и «забери фрейм». PMActors получают
|
||||
свои диапазоны из PMM на этапе инициализации и больше к нему не
|
||||
обращаются.
|
||||
258
kernel/docs/memory/pm-actor.md
Normal file
258
kernel/docs/memory/pm-actor.md
Normal file
@@ -0,0 +1,258 @@
|
||||
# PM Actor: распределённый менеджер физической памяти
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
**PMActor** (Physical Memory Actor) — это изолированный агент, который
|
||||
владеет фиксированным диапазоном физической памяти и управляет им через
|
||||
BuddyAllocator. Он следует **модели акторов**:
|
||||
|
||||
- У каждого актора есть **почтовый ящик** (MPSC queue).
|
||||
- Другие компоненты **отправляют ему сообщения** (Allocate, Free, Carve).
|
||||
- Актёр **обрабатывает сообщения**, когда получает CPU time.
|
||||
- Актёр **отправляет ответы** через PM Router.
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────┐
|
||||
│ PMActor #1 │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────────────────┐ │
|
||||
│ │ MPSC Inbox (PMActorQueue) │ │
|
||||
│ │ [Alloc] [Free] [Carve] [Alloc] [Free] ... │ │
|
||||
│ └──────────────────────┬───────────────────────┘ │
|
||||
│ │ pop │
|
||||
│ ┌──────────────────────▼───────────────────────┐ │
|
||||
│ │ process_messages() │ │
|
||||
│ │ ┌────────────┐ ┌──────────┐ ┌──────────┐ │ │
|
||||
│ │ │handle_alloc│ │handle_free│ │handle_carve│ │ │
|
||||
│ │ └──────┬─────┘ └────┬─────┘ └─────┬────┘ │ │
|
||||
│ │ ▼ ▼ ▼ │ │
|
||||
│ │ ┌────────────────────────────────────────┐ │ │
|
||||
│ │ │ BuddyAllocator │ │ │
|
||||
│ │ │ [диапазон: 0x4000_0000 — 0x4000_xxxx] │ │ │
|
||||
│ │ └────────────────────────────────────────┘ │ │
|
||||
│ └────────────────────────┬──────────────────────┘ │
|
||||
│ │ Vec<PMResponse> │
|
||||
│ ▼ │
|
||||
│ Через PMRouter.route_responses() │
|
||||
└──────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Структура PMActor
|
||||
|
||||
```rust
|
||||
pub struct PMActor {
|
||||
pub actor_id: u64, // Уникальный ID
|
||||
pub root_untyped: Capability, // Capability на весь диапазон
|
||||
pub managed_range: (PhysAddr, PhysAddr), // [start, end)
|
||||
queue: PMActorQueue, // MPSC inbox
|
||||
buddy: BuddyAllocator, // Buddy-аллокатор
|
||||
}
|
||||
```
|
||||
|
||||
- `actor_id` — используется в `token_sig` для уникальной идентификации.
|
||||
- `root_untyped` — Strong capability на весь диапазон (корень дерева).
|
||||
- `managed_range` — физические границы (начало, конец).
|
||||
- `queue` — lock-free MPSC кольцевой буфер.
|
||||
- `buddy` — аллокатор для этого диапазона.
|
||||
|
||||
## PMActorQueue — lock-free MPSC кольцевой буфер
|
||||
|
||||
```rust
|
||||
pub struct PMActorQueue {
|
||||
buffer: [AtomicU64; QUEUE_SIZE], // 1024 entry
|
||||
_pad0: [u8; 64],
|
||||
head: AtomicUsize, // читатель (актор)
|
||||
_pad1: [u8; 64],
|
||||
tail: AtomicUsize, // писатели (производители)
|
||||
}
|
||||
```
|
||||
|
||||
### Зачем padding?
|
||||
|
||||
`_pad0` и `_pad1` размером 64 байта — это размер кэш-линии x86-64.
|
||||
Разделение head и tail по разным кэш-линиям предотвращает **false sharing**
|
||||
(когда два ядра пишут в разные переменные, но они на одной кэш-линии).
|
||||
|
||||
### send(req) — Multi-Producer
|
||||
|
||||
```rust
|
||||
pub fn send(&self, req: PMRequest) -> Result<(), &'static str> {
|
||||
let packed = req.pack();
|
||||
let mut tail = self.tail.load(Ordering::Relaxed);
|
||||
loop {
|
||||
let head = self.head.load(Ordering::Acquire);
|
||||
if tail.wrapping_sub(head) >= QUEUE_SIZE {
|
||||
return Err("Queue overflow");
|
||||
}
|
||||
// CAS: пытаемся захватить слот
|
||||
match self.tail.compare_exchange_weak(tail, tail+1, AcqRel, Relaxed) {
|
||||
Ok(_) => {
|
||||
self.buffer[tail & QUEUE_MASK].store(packed, Release);
|
||||
return Ok(());
|
||||
}
|
||||
Err(actual) => tail = actual,
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `wrapping_sub`: кольцевой буфер, переполнение при разнице >= QUEUE_SIZE.
|
||||
- `compare_exchange_weak`: позволяет нескольким продюсерам соревноваться.
|
||||
- Store с `Release`: гарантирует, что данные видны потребителю.
|
||||
|
||||
### pop() — Single-Consumer
|
||||
|
||||
```rust
|
||||
pub fn pop(&self) -> Option<PMRequest> {
|
||||
let head = self.head.load(Ordering::Relaxed);
|
||||
let tail = self.tail.load(Ordering::Acquire);
|
||||
if head == tail { return None; }
|
||||
let packed = self.buffer[head & QUEUE_MASK].load(Ordering::Acquire);
|
||||
self.head.store(head + 1, Ordering::Release);
|
||||
Some(PMRequest::unpack(packed))
|
||||
}
|
||||
```
|
||||
|
||||
- Только актор вызывает pop — никакой конкуренции.
|
||||
- Acquire на tail: видим последнюю запись продюсера.
|
||||
|
||||
## Message Packing
|
||||
|
||||
Каждое сообщение упаковывается в одно `u64`:
|
||||
|
||||
```
|
||||
Bit 63:56 OPCODE (8 bits) 1=Alloc, 2=Free, 3=Carve
|
||||
Bit 55:40 CHANNEL_ID (16 bits) маршрутизация ответа; 0 = discard
|
||||
Bit 39:20 ARG1 (20 bits) size_pages/local_frame_idx/offset_pages
|
||||
Bit 19: 0 ARG2 (20 bits) token_sig (low 20 bits) / size_pages / order
|
||||
```
|
||||
|
||||
Максимум: 2^20 = 1 048 576 страниц = 4 GiB на одну аллокацию.
|
||||
|
||||
## Типы сообщений (PMRequest)
|
||||
|
||||
```rust
|
||||
pub enum PMRequest {
|
||||
Allocate {
|
||||
size_pages: usize, // сколько страниц
|
||||
token_sig: u32, // тег capability
|
||||
channel_id: u16, // куда отправить ответ
|
||||
},
|
||||
Free {
|
||||
local_frame_idx: usize, // относительный индекс
|
||||
order: usize, // buddy order
|
||||
},
|
||||
Carve {
|
||||
offset_pages: usize, // смещение от начала
|
||||
size_pages: usize, // размер
|
||||
channel_id: u16, // куда отправить ответ
|
||||
},
|
||||
None,
|
||||
}
|
||||
```
|
||||
|
||||
## PMResponse — результат
|
||||
|
||||
```rust
|
||||
pub struct PMResponse {
|
||||
pub channel_id: u16,
|
||||
pub result: PMResult,
|
||||
}
|
||||
|
||||
pub enum PMResult {
|
||||
Allocated { cap: Capability, order: usize },
|
||||
OutOfMemory { size_pages: usize },
|
||||
Carved { cap: Capability },
|
||||
Freed { pages_returned: usize },
|
||||
}
|
||||
```
|
||||
|
||||
## Обработка сообщений
|
||||
|
||||
### handle_allocate(size_pages, token_sig, channel_id)
|
||||
|
||||
```rust
|
||||
fn handle_allocate(&mut self, size_pages, token_sig, channel_id) -> Option<PMResponse> {
|
||||
match self.buddy.alloc_pages(size_pages) {
|
||||
Some((rel_idx, order)) => {
|
||||
let phys = PhysAddr(self.managed_range.0.0 + rel_idx * PAGE_SIZE);
|
||||
let cap = Capability {
|
||||
object: CapObject::Memory { phys, size_pages: 1 << order },
|
||||
rights: CapRights::READ | WRITE | GRANT,
|
||||
relation: Relation::Strong,
|
||||
token_sig: (token_sig as u64) ^ self.actor_id ^ (rel_idx as u64),
|
||||
};
|
||||
Some(PMResponse { channel_id, result: Allocated { cap, order } })
|
||||
}
|
||||
None => {
|
||||
Some(PMResponse { channel_id, result: OutOfMemory { size_pages } })
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Детали:**
|
||||
- `rel_idx` — относительный индекс в buddy (от 0 до total_pages).
|
||||
- Физический адрес: `managed_range.start + rel_idx * 4096`.
|
||||
- `token_sig` — XOR от token_sig запроса, actor_id и rel_idx для уникальности.
|
||||
- Размер в capability: `1 << order` (округляется до степени двойки).
|
||||
|
||||
### handle_free(local_frame_idx, order)
|
||||
|
||||
```rust
|
||||
fn handle_free(&mut self, local_frame_idx, order) -> Option<PMResponse> {
|
||||
// Валидация границ
|
||||
let block_end = local_frame_idx.saturating_add(1usize << order);
|
||||
if block_end > self.buddy.total_pages() {
|
||||
debug_assert!(false, "Out of range");
|
||||
return None;
|
||||
}
|
||||
self.buddy.free(local_frame_idx, order);
|
||||
None // Free не требует ответа
|
||||
}
|
||||
```
|
||||
|
||||
### handle_carve(offset_pages, size_pages, channel_id)
|
||||
|
||||
Carve — выделение под-диапазона без участия buddy-аллокатора.
|
||||
Используется для статических размещений (framebuffer alias).
|
||||
|
||||
```rust
|
||||
fn handle_carve(&mut self, offset_pages, size_pages, channel_id) -> Option<PMResponse> {
|
||||
let range_pages = self.buddy.total_pages();
|
||||
// Проверка границ
|
||||
if offset_pages >= range_pages || size_pages == 0 || ... {
|
||||
return Some(PMResponse { channel_id, result: OutOfMemory { size_pages } });
|
||||
}
|
||||
let phys = PhysAddr(self.managed_range.0.0 + offset_pages * PAGE_SIZE);
|
||||
let cap = Capability {
|
||||
object: CapObject::Memory { phys, size_pages },
|
||||
rights: CapRights::READ | WRITE | GRANT,
|
||||
relation: Relation::Strong,
|
||||
token_sig: self.actor_id ^ offset_pages,
|
||||
};
|
||||
Some(PMResponse { channel_id, result: Carved { cap } })
|
||||
}
|
||||
```
|
||||
|
||||
## Ballooning hook (TODO)
|
||||
|
||||
Когда `BuddyAllocator::alloc_pages` возвращает `None`, актор генерирует
|
||||
`OutOfMemory`. Будущая подсистема ballooning будет перехватывать этот
|
||||
ответ до того, как он дойдёт до запросившего, и договариваться о передаче
|
||||
памяти от соседнего актора.
|
||||
|
||||
## Владение и capabilities
|
||||
|
||||
- PMActor держит Strong capability на весь свой диапазон.
|
||||
- Все выделенные или вырезанные под-диапазоны — тоже Strong (дочерние).
|
||||
- При освобождении через Free buddy-дерево объединяет блоки.
|
||||
- Актёр **никогда** не обращается к глобальному BitmapPMM.
|
||||
|
||||
## Максимальное количество сообщений за вызов
|
||||
|
||||
```rust
|
||||
const MAX_MESSAGES_PER_CALL: usize = 64;
|
||||
```
|
||||
|
||||
Ограничение предотвращает голодание ядра, когда inbox глубок.
|
||||
189
kernel/docs/memory/pm-router.md
Normal file
189
kernel/docs/memory/pm-router.md
Normal file
@@ -0,0 +1,189 @@
|
||||
# PM Router: Lock-Free маршрутизация ответов
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
**PM Router** — это центральный коммутатор, соединяющий исполнителей
|
||||
запросов (PMActors) с потребителями результатов (процессы/потоки).
|
||||
|
||||
- Он предоставляет **65536 каналов** для асинхронной коммуникации.
|
||||
- Каналы **lock-free**: alloc/release канала — CAS на атомарных счётчиках.
|
||||
- Исключает блокировки между продюсером и консьюмером.
|
||||
|
||||
```
|
||||
Процесс A PMActor #1
|
||||
│ │
|
||||
│ alloc_channel() │
|
||||
│ ─────────► router ──────► │
|
||||
│ │
|
||||
│ submit_request(Alloc) │
|
||||
│ ────────────────────────► │
|
||||
│ │ process_messages()
|
||||
│ │ ──────► buddy.alloc()
|
||||
│ │
|
||||
│ ◄─────────────── │
|
||||
│ route_responses([Resp]) │
|
||||
│ router │
|
||||
│ │
|
||||
│ wait_for_response(ch) │
|
||||
│ ◄═══ RESULT ════ │
|
||||
│ │
|
||||
│ free_channel (неявно) │
|
||||
```
|
||||
|
||||
## Структура канала
|
||||
|
||||
```rust
|
||||
#[repr(align(64))]
|
||||
pub struct Channel {
|
||||
state: AtomicU8, // FREE=0, PENDING=1, READY=2
|
||||
next_free: AtomicU16, // указатель в стеке свободных каналов
|
||||
result: UnsafeCell<Option<PMResult>>, // ячейка результата
|
||||
}
|
||||
```
|
||||
|
||||
- `align(64)` — каждая кэш-линия содержит ровно один канал.
|
||||
- `state` — конечный автомат: FREE → PENDING → READY → FREE.
|
||||
- `UnsafeCell` — потому что write происходит из route_responses,
|
||||
read — из wait_for_response. Синхронизация через state.
|
||||
|
||||
## Структура PMRouter
|
||||
|
||||
```rust
|
||||
pub struct PMRouter {
|
||||
channels: Box<[Channel]>, // 65536 каналов, Box<[T]> в куче
|
||||
free_head: AtomicU16, // стек свободных каналов
|
||||
}
|
||||
```
|
||||
|
||||
### Стек свободных каналов
|
||||
|
||||
Изначально все каналы свободны, `free_head = 1` (канал 0 зарезервирован
|
||||
как «discard» — ответы на канал 0 игнорируются).
|
||||
|
||||
```
|
||||
free_head ──► channel[1].next_free = 2
|
||||
channel[2].next_free = 3
|
||||
channel[3].next_free = 4
|
||||
...
|
||||
channel[65535].next_free = 0 (NULL)
|
||||
```
|
||||
|
||||
## Операции
|
||||
|
||||
### alloc_channel() — выделить канал
|
||||
|
||||
```rust
|
||||
pub fn alloc_channel(&self) -> Option<u16> {
|
||||
let mut head = self.free_head.load(Acquire);
|
||||
loop {
|
||||
if head == 0 { return None; } // нет свободных
|
||||
let next = self.channels[head].next_free.load(Relaxed);
|
||||
// CAS: free_head = head → next
|
||||
match self.free_head.compare_exchange_weak(head, next, AcqRel, Acquire) {
|
||||
Ok(_) => {
|
||||
self.channels[head].state.store(STATE_PENDING, Release);
|
||||
return Some(head);
|
||||
}
|
||||
Err(new) => head = new,
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Lock-free: CAS на `free_head` позволяет нескольким продюсерам
|
||||
конкурировать без блокировок.
|
||||
|
||||
### route_responses(responses) — запись результатов
|
||||
|
||||
```rust
|
||||
pub fn route_responses(&self, responses: Vec<PMResponse>) {
|
||||
for resp in responses {
|
||||
if resp.channel_id == 0 { continue; }
|
||||
let channel = &self.channels[resp.channel_id as usize];
|
||||
unsafe { *channel.result.get() = Some(resp.result); }
|
||||
channel.state.store(STATE_READY, Release);
|
||||
// TODO: Focus Mode — пробуждение ожидающего потока
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### wait_for_response(id) — ожидание результата
|
||||
|
||||
```rust
|
||||
pub fn wait_for_response(&self, id: u16) -> PMResult {
|
||||
let channel = &self.channels[id as usize];
|
||||
// busy-wait с HLT
|
||||
while channel.state.load(Acquire) != STATE_READY {
|
||||
unsafe { asm!("hlt") }; // CPU остановка до прерывания
|
||||
}
|
||||
let result = unsafe { (*channel.result.get()).take().unwrap() };
|
||||
channel.state.store(STATE_FREE, Release);
|
||||
|
||||
// Возвращаем канал в стек свободных
|
||||
let mut head = self.free_head.load(Relaxed);
|
||||
loop {
|
||||
channel.next_free.store(head, Relaxed);
|
||||
match self.free_head.compare_exchange_weak(head, id, Release, Relaxed) {
|
||||
Ok(_) => break,
|
||||
Err(new) => head = new,
|
||||
}
|
||||
}
|
||||
result
|
||||
}
|
||||
```
|
||||
|
||||
### request_and_wait — синхронная обёртка
|
||||
|
||||
```rust
|
||||
pub fn request_and_wait<F>(actor: &PMActor, req_builder: F) -> PMResult
|
||||
where
|
||||
F: FnOnce(u16) -> PMRequest
|
||||
{
|
||||
let router = get_router();
|
||||
let channel_id = router.alloc_channel().expect("OOM");
|
||||
let req = req_builder(channel_id);
|
||||
actor.submit_request(req).expect("Inbox full");
|
||||
router.wait_for_response(channel_id)
|
||||
}
|
||||
```
|
||||
|
||||
## Глобальный экземпляр
|
||||
|
||||
```rust
|
||||
static ROUTER: GlobalRouter = GlobalRouter {
|
||||
is_ready: AtomicBool::new(false),
|
||||
inner: UnsafeCell::new(None),
|
||||
};
|
||||
```
|
||||
|
||||
Double-checked initialization:
|
||||
1. `init()` — сетап всех каналов, `is_ready = true`.
|
||||
2. `get_router()` — проверяет `is_ready`, затем `unwrap_unchecked()`.
|
||||
|
||||
**Безопасность**: `is_ready` устанавливается один раз и никогда не
|
||||
очищается, поэтому TOCTOU между проверкой и unwrap — безопасен.
|
||||
|
||||
## dispatch() — утилита
|
||||
|
||||
```rust
|
||||
pub fn dispatch(responses: Vec<PMResponse>) {
|
||||
get_router().route_responses(responses);
|
||||
}
|
||||
```
|
||||
|
||||
## Ключевые свойства
|
||||
|
||||
| Свойство | Значение |
|
||||
|----------|----------|
|
||||
| Количество каналов | 65536 |
|
||||
| Размер канала | 64 байта (1 cache line) |
|
||||
| alloc_channel | Lock-free, O(1) |
|
||||
| route_responses | O(N), N = количество ответов |
|
||||
| wait_for_response | Lock-free + HLT |
|
||||
| Состояния | FREE → PENDING → READY → FREE |
|
||||
|
||||
## Где используется
|
||||
|
||||
- В `kmain()`: тестирование PMActor через Router.
|
||||
- Будущие IPC и процесс-менеджер будут использовать Router для
|
||||
асинхронного взаимодействия с PMActors.
|
||||
267
kernel/docs/memory/vmm.md
Normal file
267
kernel/docs/memory/vmm.md
Normal file
@@ -0,0 +1,267 @@
|
||||
# Virtual Memory Manager: `vmm.rs`
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
**AddressSpace** — это представление виртуального адресного пространства
|
||||
процесса. Оно состоит из:
|
||||
|
||||
1. **Аппаратной части**: PML4 (Page Map Level 4) — корневая таблица страниц.
|
||||
2. **Программной части**: VMA (Virtual Memory Area) список — отсортированный
|
||||
перечень отображённых регионов с метаданными.
|
||||
|
||||
```
|
||||
AddressSpace
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ asid: u16 (PCID = Process Context ID) │
|
||||
│ pml4_phys: PhysAddr (адрес PML4) │
|
||||
│ regions: Vec<VmaRegion> (сортирован по адресу)│
|
||||
└─────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────┐
|
||||
│ PML4 (P4) │ ◄── CR3
|
||||
├───────────────┤
|
||||
│ PDE (P3) │
|
||||
├───────────────┤
|
||||
│ PTE (P2) │
|
||||
├───────────────┤
|
||||
│ PTE (P1) │ ◄── 4 KiB страницы
|
||||
└───────────────┘
|
||||
```
|
||||
|
||||
## VmaRegion — регион виртуальной памяти
|
||||
|
||||
```rust
|
||||
pub struct VmaRegion {
|
||||
pub virt_start: VirtAddr, // начало (выровнено на 4k)
|
||||
pub virt_end: VirtAddr, // конец (эксклюзивный)
|
||||
pub flags: VmaFlags, // READ, WRITE, EXEC, USER, LAZY, SHARED, COW...
|
||||
pub cap_token: u64, // токен capability (для revoke)
|
||||
pub backing: VmaBacking, // откуда берутся физические страницы
|
||||
}
|
||||
```
|
||||
|
||||
### VmaBacking — источник физических страниц
|
||||
|
||||
```rust
|
||||
pub enum VmaBacking {
|
||||
Anonymous(Vec<Option<PhysAddr>>),
|
||||
// ^ Owned: каждая страница выделена из PMM под этот VMA.
|
||||
// Option: None = lazy (страница ещё не выделена).
|
||||
|
||||
Physical(PhysAddr),
|
||||
// ^ Не-owned: всё отображение — непрерывный физический диапазон.
|
||||
// Frame не освобождаются при unmap.
|
||||
|
||||
Shared { owner_cap: u64, phys_base: PhysAddr },
|
||||
// ^ Zero-copy shared mapping: чужие физические страницы.
|
||||
// Frame не освобождаются при unmap.
|
||||
// При revoke owner_cap — все Shared VMA с этим токеном удаляются.
|
||||
}
|
||||
```
|
||||
|
||||
### VmaFlags
|
||||
|
||||
```rust
|
||||
pub struct VmaFlags: u32 {
|
||||
const READ = 1 << 0;
|
||||
const WRITE = 1 << 1;
|
||||
const EXEC = 1 << 2;
|
||||
const USER = 1 << 3;
|
||||
const LAZY = 1 << 4; // Demand-paging: page fault = alloc frame
|
||||
const SHARED = 1 << 5; // Zero-copy shared mapping
|
||||
const PINNED = 1 << 6; // Защищён от revoke
|
||||
const NOCACHE = 1 << 7; // Cache disabled (для MMIO)
|
||||
const MMIO = 1 << 8; // Memory-mapped I/O
|
||||
const COW = 1 << 9; // Copy-on-Write active
|
||||
}
|
||||
```
|
||||
|
||||
`to_page_flags()` транслирует VmaFlags в PageTableFlags:
|
||||
- PRESENT всегда
|
||||
- WRITE → WRITABLE
|
||||
- EXEC → NO_EXECUTE (инвертировано!)
|
||||
- LAZY → PRESENT без фрейма (fault отлавливается)
|
||||
- COW → COW (кастомный бит 9)
|
||||
|
||||
## ASID Allocator — выделение PCID
|
||||
|
||||
x86-64 PCID (Process Context Identifier) — 12-битный идентификатор (0-4095).
|
||||
|
||||
```rust
|
||||
struct AsidAllocator {
|
||||
bitmap: [u32; 128], // 4096 бит
|
||||
next_hint: u16, // хинт для амортизированного O(1)
|
||||
}
|
||||
```
|
||||
|
||||
- ASID 0: kernel (no PCID tagging) — всегда занят.
|
||||
- ASID 4095: reserved by Intel spec — всегда занят.
|
||||
- ASID 1–4094: пользовательские.
|
||||
|
||||
`alloc()` — Two-pass: от hint до 4094, затем от 1 до hint.
|
||||
Пропускает полностью занятые 32-битные слова.
|
||||
|
||||
## Основные операции AddressSpace
|
||||
|
||||
### `new()` — создание пустого пространства
|
||||
|
||||
1. Аллоцирует фрейм из PMM для PML4.
|
||||
2. Обнуляет его (все entry = not present).
|
||||
3. Выделяет ASID из ASID_ALLOC.
|
||||
4. Пустой список regions.
|
||||
|
||||
### `from_active(pml4_phys, asid)` — обёртка существующего PML4
|
||||
|
||||
Используется для kernel space (ASID 0).
|
||||
|
||||
### `map_region(virt, phys, size, flags, cap_token)` — отображение
|
||||
|
||||
```rust
|
||||
pub fn map_region(&mut self, virt: VirtAddr, phys: Option<PhysAddr>,
|
||||
size: u64, flags: VmaFlags, cap_token: u64) -> Result<VirtAddr, VmError>
|
||||
```
|
||||
|
||||
- `phys = Some(base)`: Physical mapping — непрерывный диапазон.
|
||||
- `phys = None + LAZY`: Demand-paging — фреймы выделяются при fault'е.
|
||||
- `phys = None + !LAZY`: Eager alloc — все фреймы выделяются сейчас.
|
||||
|
||||
Создаёт VmaRegion, вставляет в отсортированный список.
|
||||
|
||||
### `map_shared(virt, phys_base, page_count, flags, owner_cap)` — zero-copy
|
||||
|
||||
Как map_region, но:
|
||||
- Не копирует фреймы (не владеет ими).
|
||||
- Всегда eagerly отображает (адреса известны).
|
||||
- Записывает `cap_token = owner_cap` для revoke.
|
||||
|
||||
### `unmap_region(virt)` — удаление VMA
|
||||
|
||||
1. Находит VMA по адресу.
|
||||
2. Удаляет из списка.
|
||||
3. Вызывает `do_unmap(pml4, region)`:
|
||||
- unmap_page для каждой страницы.
|
||||
- Если backing owns_frames (Anonymous) — free_frame.
|
||||
4. TLB flush.
|
||||
|
||||
### `handle_fault(fault_addr, write)` — Page Fault handler
|
||||
|
||||
```rust
|
||||
pub fn handle_fault(&mut self, fault_addr: VirtAddr, write: bool) -> Result<(), VmError>
|
||||
```
|
||||
|
||||
1. Находит VMA, содержащий fault_addr.
|
||||
2. Проверяет права (write + !WRITE = PermissionDenied).
|
||||
3. Проверяет COW:
|
||||
- Если write + COW: аллоцирует новый фрейм, копирует данные,
|
||||
заменяет PTE, освобождает старый фрейм.
|
||||
4. Проверяет LAZY:
|
||||
- Если LAZY + page не выделена: alloc_frame, обнуление, map_page.
|
||||
5. Иначе: UnexpectedFault → panic.
|
||||
|
||||
### `clone_for_fork(child_cap_token)` — fork адресного пространства
|
||||
|
||||
1. Создаёт пустое AddressSpace::new().
|
||||
2. Для каждого VMA родителя:
|
||||
- Physical/Shared: просто копирует PTE (та же физическая память).
|
||||
- Anonymous (owned): COW.
|
||||
* Устанавливает COW флаг на родительские PTE.
|
||||
* Убирает WRITABLE с родительских PTE.
|
||||
* Отображает те же фреймы в дочернее пространство (без WRITABLE).
|
||||
* При записи → handle_fault → copy-on-write.
|
||||
|
||||
### `revoke_by_token(cap_token)` / `do_revoke_by_token(cap_token)` — отзыв
|
||||
|
||||
Удаляет все нем-PINNED VMA с заданным cap_token.
|
||||
- Собирает индексы регионов, соответствующих токену.
|
||||
- Удаляет в обратном порядке (чтобы индексы не сдвигались).
|
||||
- TLB flush.
|
||||
|
||||
### `process_pending_revocations()` — обработка очереди отзыва
|
||||
|
||||
Дренирует глобальную `MMU_REVOCATION_QUEUE` и revoke-ит каждый токен.
|
||||
Вызывается перед handle_fault при page fault'е, чтобы избежать deadlock
|
||||
с capability subsystem.
|
||||
|
||||
## TLB Management
|
||||
|
||||
### `local_tlb_flush_asid(asid)` — сброс TLB для одного ASID
|
||||
|
||||
```rust
|
||||
fn local_tlb_flush_asid(asid: u16) {
|
||||
if INVPCID_SUPPORTED {
|
||||
// INVPCID type 1 (single-context): сбрасывает только
|
||||
// TLB entries, помеченные данным PCID.
|
||||
asm!("invpcid {ty}, [{desc}]", ...);
|
||||
} else {
|
||||
// Fallback: полный сброс TLB (все контексты)
|
||||
tlb_flush_all();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `tlb_flush_asid(asid)` — TLB shootdown (SMP)
|
||||
|
||||
```rust
|
||||
pub fn tlb_flush_asid(asid: u16) {
|
||||
local_tlb_flush_asid(asid); // всегда сбрасываем локальный
|
||||
|
||||
let target_mask = active_cpus & !(1 << current_core);
|
||||
if target_mask == 0 { return; }
|
||||
|
||||
// Lock SHOOTDOWN_LOCK → записываем ASID → broadcast IPI → wait ACK
|
||||
SHOOTDOWN_ASID.store(asid);
|
||||
SHOOTDOWN_ACK.store(0);
|
||||
lapic::broadcast_ipi_exclude_self(TLB_SHOOTDOWN_VECTOR);
|
||||
|
||||
while SHOOTDOWN_ACK.load() & target_mask != target_mask {
|
||||
spin_loop();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `tlb_flush_all()` — полный сброс
|
||||
|
||||
Перезапись CR3 без PCID NOFLUSH бита.
|
||||
|
||||
## AddressSpace::activate() — контекстный переключатель
|
||||
|
||||
```rust
|
||||
pub unsafe fn activate(&self) {
|
||||
let cr3 = self.pml4_phys.0 | u64::from(self.asid) | (1u64 << 63);
|
||||
// ^ Биты 11:0 = ASID, бит 63 = NOFLUSH (не сбрасывать TLB)
|
||||
asm!("mov cr3, {0}", in(reg) cr3);
|
||||
}
|
||||
```
|
||||
|
||||
## Глобальный kernel space
|
||||
|
||||
```rust
|
||||
pub static KERNEL_SPACE: Locked<Option<AddressSpace>> = Locked::new(None);
|
||||
```
|
||||
|
||||
Инициализируется в `init_kernel_space(pml4_phys)` после создания
|
||||
первичных page tables.
|
||||
|
||||
## Drop для AddressSpace
|
||||
|
||||
При Drop:
|
||||
1. unmap всех регионов (с освобождением owned frames).
|
||||
2. free_frame(pml4_phys).
|
||||
3. free_asid(asid).
|
||||
|
||||
## VmError — типы ошибок
|
||||
|
||||
```rust
|
||||
pub enum VmError {
|
||||
OutOfMemory, // Нет фреймов в PMM
|
||||
RegionOverlap, // VMA пересекается с существующим
|
||||
RegionNotFound, // Нет VMA по адресу
|
||||
InvalidAlignment, // Адрес/размер не выровнены на 4k
|
||||
InvalidRange, // Нулевой размер или переполнение
|
||||
PermissionDenied, // Нет прав на запись
|
||||
UnexpectedFault, // Fault в не-lazy, не-COW регионе
|
||||
NonCanonical, // Некорректный адрес
|
||||
AsidExhausted, // Закончились ASID
|
||||
}
|
||||
```
|
||||
105
kernel/docs/overview/architecture.md
Normal file
105
kernel/docs/overview/architecture.md
Normal file
@@ -0,0 +1,105 @@
|
||||
# Архитектура ядра Elyz
|
||||
|
||||
## Высокоуровневая схема
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ kmain() — точка входа │
|
||||
│ (инициализация: Limine, PMM, LAPIC, IDT, VMM, PM Router, Caps) │
|
||||
└──────────────┬──────────────────────────────────────┬───────────────┘
|
||||
│ │
|
||||
┌──────────▼──────────┐ ┌─────────▼──────────────┐
|
||||
│ memory subsystem │ │ capability system │
|
||||
│ │ │ │
|
||||
│ ┌───────────────┐ │ │ ┌──────────────────┐ │
|
||||
│ │ BitmapPMM │ │ │ │ CapDescriptor │ │
|
||||
│ │ (глобальный) │ │ │ │ (права + объект) │ │
|
||||
│ └───────┬───────┘ │ │ └────────┬─────────┘ │
|
||||
│ │ │ │ │ │
|
||||
│ ┌───────▼───────┐ │ │ ┌────────▼─────────┐ │
|
||||
│ │ PM Actor + │ │ │ │ CNode │ │
|
||||
│ │ BuddyAlloc │ │ │ │ (таблица caps) │ │
|
||||
│ └───────┬───────┘ │ │ └──────────────────┘ │
|
||||
│ │ │ │ │
|
||||
│ ┌───────▼───────┐ │ └────────────────────────┘
|
||||
│ │ PM Router │ │
|
||||
│ │ (каналы) │ │
|
||||
│ └───────┬───────┘ │
|
||||
│ │ │
|
||||
│ ┌───────▼───────┐ │
|
||||
│ │ AddressSpace │ │
|
||||
│ │ (VMM, PML4) │ │
|
||||
│ └───────────────┘ │
|
||||
└─────────────────────┘
|
||||
|
||||
┌──────────────────────┐ ┌──────────────────────┐
|
||||
│ cpu subsystem │ │ debug subsystem │
|
||||
│ ┌────────────────┐ │ │ ┌─────────────────┐ │
|
||||
│ │ IDT + Interrupt│ │ │ │ Serial port │ │
|
||||
│ │ Handlers │ │ │ │ (COM1) │ │
|
||||
│ └────────────────┘ │ │ └─────────────────┘ │
|
||||
│ ┌────────────────┐ │ │ ┌─────────────────┐ │
|
||||
│ │ Local APIC │ │ │ │ Log macros │ │
|
||||
│ └────────────────┘ │ │ │ (info!/warn!) │ │
|
||||
└──────────────────────┘ │ └─────────────────┘ │
|
||||
└──────────────────────┘
|
||||
```
|
||||
|
||||
## Порядок инициализации в `kmain()`
|
||||
|
||||
1. **Limine requests** — статические структуры, сообщающие bootloader'у,
|
||||
что ядру нужны: framebuffer, memory map, HHDM offset, адрес ядра.
|
||||
2. **BASE_REVISION проверка** — убеждаемся, что Limine совместим.
|
||||
3. **HHDM инициализация** — сохраняем Higher-Half Direct Map offset.
|
||||
4. **Framebuffer/TTY** — создаём консоль для вывода на экран.
|
||||
5. **Serial port** — инициализируем COM1 для отладки.
|
||||
6. **Early IDT** — загружаем базовые обработчики исключений (0–31).
|
||||
7. **BitmapPMM::init()** — сканируем карту памяти, строим битовую карту.
|
||||
8. **LAPIC::init()** — настраиваем Local APIC.
|
||||
9. **Создание P4** — аллоцируем фрейм для корневой таблицы страниц.
|
||||
10. **Identity map** — отображаем всю физическую память в HHDM
|
||||
и (для non-reserved) в идентичное отображение.
|
||||
11. **Map kernel** — отображаем образ ядра.
|
||||
12. **Activate P4** — загружаем новую таблицу страниц в CR3.
|
||||
13. **Heap init** — резервируем 8 MiB для slab-аллокатора.
|
||||
14. **CPU features** — проверяем INVPCID.
|
||||
15. **VMM init** — регистрируем AddressSpace ядра.
|
||||
16. **IDT final** — перезагружаем IDT с Page Fault и TLB shootdown.
|
||||
17. **PM Router init** — создаём 65536 каналов.
|
||||
18. **STI** — разрешаем прерывания.
|
||||
19. **Capability тесты** — тестируем CNode, mint, revoke.
|
||||
20. **PMActor тесты** — создаём актор, тестируем Alloc/Free/Carve/OOM.
|
||||
21. **Buddy тесты** — прямое тестирование аллокатора.
|
||||
22. **PMM stress test** — alloc/free 64 фреймов.
|
||||
23. **HCF** — гасим CPU.
|
||||
|
||||
## Модульная структура
|
||||
|
||||
```
|
||||
src/
|
||||
├── main.rs — точка входа, инициализация, тесты
|
||||
├── cap/ — capability система
|
||||
│ ├── mod.rs — CNode (таблица дескрипторов)
|
||||
│ ├── descriptor.rs — типы Capability, CapObject, CapRights, Relation
|
||||
│ └── object.rs — KernelObject (ref-counted)
|
||||
├── cpu/ — подсистема CPU
|
||||
│ ├── mod.rs — реэкспорт модулей
|
||||
│ ├── idt.rs — IDT entry/table/ptr
|
||||
│ ├── interrupts.rs — обработчики исключений, TLB shootdown
|
||||
│ └── lapic.rs — Local APIC драйвер
|
||||
├── debug/ — подсистема отладки
|
||||
│ ├── mod.rs — LogLevel, log!/info!/warn! макросы
|
||||
│ └── serial.rs — COM1 serial port драйвер
|
||||
├── events.rs — RevocationQueue (lock-free кольцевой буфер)
|
||||
├── tty.rs — Framebuffer console (PSF2 шрифты)
|
||||
├── mem/ — подсистема памяти
|
||||
│ ├── mod.rs — реэкспорт модулей
|
||||
│ ├── address.rs — PhysAddr, VirtAddr, HHDM offset
|
||||
│ ├── allocator.rs — SlabAllocator + Locked<T> + глобальный аллокатор
|
||||
│ ├── buddy.rs — BuddyAllocator (intrusive list, O(1))
|
||||
│ ├── paging.rs — PageTable, 4-level page walk
|
||||
│ ├── pmm.rs — BitmapPMM (tree bitmap с L1 ускорением)
|
||||
│ ├── pm_manages.rs — PMActor + PMActorQueue (MPSC)
|
||||
│ ├── pm_router.rs — PMRouter (65536 lock-free каналов)
|
||||
│ └── vmm.rs — AddressSpace, VMA, COW, fork, TLB shootdown
|
||||
```
|
||||
72
kernel/docs/overview/introduction.md
Normal file
72
kernel/docs/overview/introduction.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# Введение в ядро Elyz (LISA)
|
||||
|
||||
## Концептуальная модель
|
||||
|
||||
Elyz (LISA) — это модульное, capability-ориентированное ядро для x86-64,
|
||||
спроектированное вокруг следующих архитектурных принципов:
|
||||
|
||||
### 1. Capability-ориентированная безопасность
|
||||
|
||||
Вместо традиционной модели «всё или ничего» (ring 0 vs ring 3), Elyz
|
||||
использует систему **capabilities** (дескрипторов прав). Каждый дескриптор
|
||||
представляет собой *неподделываемый токен*, дающий доступ к конкретному
|
||||
объекту ядра (фрейму памяти, CNode, PMActor) с определёнными правами.
|
||||
|
||||
- Capability — это не просто число; это структура с полями `object`, `rights`,
|
||||
`relation`, `token_sig`.
|
||||
- Capability можно *создавать* (mint) с урезанными правами от родительского
|
||||
дескриптора.
|
||||
- Capability можно *отозвать* (revoke), что каскадно уничтожает всех потомков.
|
||||
- Система гарантирует, что access — это всегда наличие capability.
|
||||
|
||||
### 2. Многоуровневое управление памятью
|
||||
|
||||
Управление физической памятью разделено на три уровня:
|
||||
|
||||
| Уровень | Компонент | Ответственность |
|
||||
|---------|-----------|-----------------|
|
||||
| 1 (глобальный) | `BitmapPMM` | Владение всей физической памятью, аллокация/освобождение по 4KiB фреймам. Инициализируется из карты памяти bootloader'а. |
|
||||
| 2 (распределённый) | `PMActor` | Актёр физической памяти — владеет *диапазоном* физической памяти, использует `BuddyAllocator` для аллокации внутри этого диапазона. Каждый PMActor — изолированный аллокатор. |
|
||||
| 3 (виртуальный) | `AddressSpace` (VMM) | Управляет виртуальными адресными пространствами (PML4 + VMA-деревья). Использует PMM для аллокации PT-страниц и lazy demand-paging. |
|
||||
|
||||
Эти уровни связывает **PM Router** — lock-free система каналов для асинхронной
|
||||
пересылки запросов и ответов между исполнителями.
|
||||
|
||||
### 3. Асинхронная модель акторов
|
||||
|
||||
PMActor следует модели CSP (Communicating Sequential Processes):
|
||||
|
||||
- Каждый актор имеет lock-free MPSC-очередь входящих сообщений.
|
||||
- Потоки/процессы отправляют запросы (Alloc/Free/Carve) в очередь актора.
|
||||
- Когда актор получает CPU time, он вызывает `process_messages()` и
|
||||
отправляет ответы через PM Router.
|
||||
- PM Router использует 65536 предварительно выделенных каналов для
|
||||
lock-free маршрутизации результатов.
|
||||
|
||||
### 4. Виртуальная память с Copy-on-Write
|
||||
|
||||
`AddressSpace` поддерживает:
|
||||
- **Lazy demand-paging**: физическая страница выделяется только при
|
||||
page fault'е.
|
||||
- **Copy-on-Write (COW)**: при fork'е адресного пространства страницы
|
||||
разделяются между родителем и потомком; копирование происходит
|
||||
при первой записи.
|
||||
- **Zero-copy shared mappings**: физические страницы, принадлежащие
|
||||
capability, могут быть отображены в другое адресное пространство
|
||||
без копирования. Владелец страниц — capability, а не VMA.
|
||||
|
||||
### 5. Отзыв (Revocation) через lock-free очередь
|
||||
|
||||
Система отзыва дескрипторов использует глобальную lock-free кольцевую
|
||||
очередь `MMU_REVOCATION_QUEUE`. При отзыве capability:
|
||||
1. Токен capability помещается в очередь.
|
||||
2. При следующем page fault'е VMM обрабатывает накопленные отзывы.
|
||||
3. Все VMA, связанные с отозванным токеном, аннулируются.
|
||||
|
||||
### 6. Минималистичный дизайн
|
||||
|
||||
- `#![no_std]` — никакой стандартной библиотеки.
|
||||
- `#![no_main]` — точка входа `kmain`.
|
||||
- Все структуры данных, аллокаторы и драйверы написаны с нуля.
|
||||
- Единственные внешние зависимости: `limine` (bootloader протокол),
|
||||
`embedded-graphics` (шрифты), `bitflags`.
|
||||
150
kernel/docs/tty.md
Normal file
150
kernel/docs/tty.md
Normal file
@@ -0,0 +1,150 @@
|
||||
# Framebuffer Console: `tty.rs`
|
||||
|
||||
## Назначение
|
||||
|
||||
Драйвер графического вывода текста на framebuffer. Использует
|
||||
PSF2 (PC Screen Font 2) шрифты для отрисовки символов.
|
||||
|
||||
Является основным визуальным каналом вывода ядра (через info! макрос).
|
||||
|
||||
## Структура Console
|
||||
|
||||
```rust
|
||||
pub struct Console<'a> {
|
||||
framebuffer: &'a Framebuffer<'a>, // ссылка на Limine framebuffer
|
||||
font: &'static [u8], // PSF2 шрифт
|
||||
pub x: usize, // текущая позиция X (колонка)
|
||||
pub y: usize, // текущая позиция Y (строка)
|
||||
pub fg_color: u32, // цвет текста (RGB)
|
||||
pub bg_color: u32, // цвет фона (RGB)
|
||||
}
|
||||
```
|
||||
|
||||
## PSF2 Header
|
||||
|
||||
```rust
|
||||
#[repr(C, packed)]
|
||||
struct Psf2Header {
|
||||
magic: u32, // 0x864AB572 (PSF2 magic)
|
||||
version: u32,
|
||||
header_size: u32, // размер заголовка
|
||||
flags: u32,
|
||||
num_glyphs: u32, // количество глифов
|
||||
bytes_per_glyph: u32, // байт на глиф
|
||||
height: u32, // высота глифа в пикселях
|
||||
width: u32, // ширина глифа в пикселях
|
||||
}
|
||||
```
|
||||
|
||||
Шрифт — `font.psf` (8×16, встроен через `include_bytes!`).
|
||||
|
||||
## Операции
|
||||
|
||||
### clear()
|
||||
|
||||
Заполняет весь framebuffer нулями. Сбрасывает позицию курсора в (0, 0).
|
||||
|
||||
```rust
|
||||
pub fn clear(&mut self) {
|
||||
unsafe {
|
||||
core::ptr::write_bytes(fb.addr(), 0, (fb.pitch() * fb.height()) as usize);
|
||||
}
|
||||
self.x = 0;
|
||||
self.y = 0;
|
||||
}
|
||||
```
|
||||
|
||||
### scroll()
|
||||
|
||||
Сдвигает содержимое framebuffer вверх на высоту шрифта.
|
||||
Новые строки заполняются нулями.
|
||||
|
||||
```rust
|
||||
fn scroll(&mut self) {
|
||||
let shift = font_height * pitch; // смещение на 1 строку
|
||||
let size = pitch * (height - font_height); // сколько копировать
|
||||
unsafe {
|
||||
ptr::copy(addr.add(shift), addr, size);
|
||||
ptr::write_bytes(addr.add(size), 0, shift);
|
||||
}
|
||||
self.y -= font_height;
|
||||
}
|
||||
```
|
||||
|
||||
### draw_glyph(glyph_index, x, y)
|
||||
|
||||
Рисует глиф (символ) в заданной позиции.
|
||||
|
||||
```rust
|
||||
fn draw_glyph(&mut self, glyph_index: u32, x: usize, y: usize) {
|
||||
let bytes_per_line = (width + 7) / 8; // пиксели → байты
|
||||
let glyph_offset = header_size + (glyph_index * bytes_per_glyph);
|
||||
|
||||
for cy in 0..height {
|
||||
let glyph_row = self.font[glyph_offset + cy * bytes_per_line];
|
||||
for cx in 0..width {
|
||||
if (glyph_row & (0x80 >> cx)) != 0 {
|
||||
// Пиксель включён — рисуем fg_color
|
||||
let offset = ((y + cy) * fb_pitch) + ((x + cx) * 4);
|
||||
unsafe {
|
||||
fb_addr.add(offset).cast::<u32>().write_volatile(self.fg_color);
|
||||
}
|
||||
}
|
||||
// Если пиксель выключен — фон остаётся (нет очистки)
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `width=8`, `bytes_per_line=1`: каждый ряд глифа = 1 байт.
|
||||
- `(0x80 >> cx)`: проверка бита слева направо (MSB first).
|
||||
- `write_volatile`: запрещает компилятору оптимизировать запись в MMIO.
|
||||
|
||||
### write_char(c)
|
||||
|
||||
Печатает символ:
|
||||
- `\n` → переход на новую строку.
|
||||
- Другие символы → draw_glyph. Если символ > num_glyphs → глиф 0.
|
||||
- Перенос строки по ширине экрана.
|
||||
- Скроллинг при достижении низа.
|
||||
|
||||
## fmt::Write реализация
|
||||
|
||||
```rust
|
||||
impl fmt::Write for Console<'_> {
|
||||
fn write_str(&mut self, s: &str) -> fmt::Result {
|
||||
for c in s.chars() {
|
||||
self.write_char(c);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Использование в kmain()
|
||||
|
||||
```rust
|
||||
let fb = fb_res.framebuffers().next().expect("No framebuffer");
|
||||
let mut console = tty::Console::new(&fb, KERNEL_FONT);
|
||||
console.clear();
|
||||
|
||||
info!(console, "BOOT", "LIS4 Kernel Starting...");
|
||||
// ... остальные логи
|
||||
|
||||
let _ = writeln!(console, "{}", logo); // ASCII art при загрузке
|
||||
```
|
||||
|
||||
## Детали реализации
|
||||
|
||||
- **Цвета**: write_volatile для 32-битного пикселя. Формат — RGBA?
|
||||
(зависит от режима framebuffer).
|
||||
- **Фон**: не перерисовывается. Только пиксели текста.
|
||||
Для полной очистки — clear().
|
||||
- **PSF2 magic**: `0x864AB572` — проверяется в заголовке.
|
||||
- **Производительность**: draw_glyph простая, без буферизации.
|
||||
Для скроллинга — memcpy всего framebuffer.
|
||||
|
||||
## Зависимости
|
||||
|
||||
- `KERNEL_FONT: &[u8]` — встроен в main.rs через `include_bytes!("font.psf")`.
|
||||
- `limine::framebuffer::Framebuffer` — через HHDM отображённый framebuffer.
|
||||
@@ -1,13 +1,16 @@
|
||||
pub mod descriptor;
|
||||
pub mod object;
|
||||
|
||||
use alloc::vec::Vec;
|
||||
use crate::events::MMU_REVOCATION_QUEUE;
|
||||
use crate::mem::allocator::Locked;
|
||||
use alloc::vec::Vec;
|
||||
|
||||
pub use descriptor::*;
|
||||
|
||||
pub struct CNodeSlot {
|
||||
pub cap: Capability,
|
||||
pub parent_idx: Option<usize>,
|
||||
pub children: Vec<usize>,
|
||||
}
|
||||
|
||||
pub struct CNode {
|
||||
@@ -21,54 +24,118 @@ impl CNode {
|
||||
slots.push(Locked::new(CNodeSlot {
|
||||
cap: Capability::empty(),
|
||||
parent_idx: None,
|
||||
children: Vec::new(),
|
||||
}));
|
||||
}
|
||||
Self { slots }
|
||||
}
|
||||
|
||||
pub fn insert(&self, slot: usize, cap: Capability) -> Result<(), &'static str> {
|
||||
if slot >= self.slots.len() { return Err("Index out of bounds"); }
|
||||
if slot >= self.slots.len() {
|
||||
return Err("Index out of bounds");
|
||||
}
|
||||
let mut s = self.slots[slot].lock();
|
||||
s.cap = cap;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn mint(&self, src: usize, dest: usize, relation: Relation, rights: CapRights) -> Result<(), &'static str> {
|
||||
let mut slots = (self.slots[src].lock(), self.slots[dest].lock());
|
||||
let (src_slot, mut dest_slot) = (&slots.0, &mut slots.1);
|
||||
if src >= self.slots.len() || dest >= self.slots.len() {
|
||||
return Err("Index out of bounds");
|
||||
}
|
||||
|
||||
if !src_slot.cap.is_valid() { return Err("Source empty"); }
|
||||
if !src_slot.cap.rights.contains(CapRights::GRANT) { return Err("Insufficient rights to mint"); }
|
||||
if src == dest {
|
||||
return Err("Source and destination slots must be different");
|
||||
}
|
||||
|
||||
let mut _guard_low;
|
||||
let mut _guard_high;
|
||||
|
||||
let (src_slot, dest_slot) = if src < dest {
|
||||
_guard_low = self.slots[src].lock();
|
||||
_guard_high = self.slots[dest].lock();
|
||||
(&mut *_guard_low, &mut *_guard_high)
|
||||
} else {
|
||||
_guard_low = self.slots[dest].lock();
|
||||
_guard_high = self.slots[src].lock();
|
||||
(&mut *_guard_high, &mut *_guard_low)
|
||||
};
|
||||
|
||||
if !src_slot.cap.is_valid() {
|
||||
return Err("Source slot is empty");
|
||||
}
|
||||
if !src_slot.cap.rights.contains(CapRights::GRANT) {
|
||||
return Err("Insufficient rights to mint (Missing GRANT flag)");
|
||||
}
|
||||
|
||||
let final_rights = src_slot.cap.rights & rights;
|
||||
|
||||
dest_slot.children.clear();
|
||||
dest_slot.cap = src_slot.cap;
|
||||
dest_slot.cap.rights = final_rights;
|
||||
dest_slot.cap.relation = relation;
|
||||
dest_slot.parent_idx = Some(src);
|
||||
|
||||
src_slot.children.push(dest);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn revoke(&self, slot_idx: usize) {
|
||||
for i in 0..self.slots.len() {
|
||||
let mut should_clear = false;
|
||||
{
|
||||
let child = self.slots[i].lock();
|
||||
if child.parent_idx == Some(slot_idx) {
|
||||
should_clear = true;
|
||||
pub fn revoke(&self, slot_idx: usize) -> Result<(), &'static str> {
|
||||
if slot_idx >= self.slots.len() {
|
||||
return Err("Index out of bounds");
|
||||
}
|
||||
|
||||
self.revoke_internal(slot_idx);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn revoke_internal(&self, slot_idx: usize) {
|
||||
let children: Vec<usize> = {
|
||||
let slot = self.slots[slot_idx].lock();
|
||||
slot.children.clone()
|
||||
};
|
||||
|
||||
for child in children {
|
||||
let is_valid_child = {
|
||||
let c = self.slots[child].lock();
|
||||
c.parent_idx == Some(slot_idx) && c.cap.is_valid()
|
||||
};
|
||||
if is_valid_child {
|
||||
self.revoke_internal(child);
|
||||
}
|
||||
}
|
||||
|
||||
if should_clear {
|
||||
self.revoke(i);
|
||||
let mut child = self.slots[i].lock();
|
||||
child.cap = Capability::empty();
|
||||
child.parent_idx = None;
|
||||
let (token_to_revoke, parent_idx) = {
|
||||
let mut slot = self.slots[slot_idx].lock();
|
||||
let parent = slot.parent_idx;
|
||||
let token = if slot.cap.is_valid() {
|
||||
let t = slot.cap.token_sig;
|
||||
slot.cap = Capability::empty();
|
||||
slot.parent_idx = None;
|
||||
slot.children.clear();
|
||||
Some(t)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
(token, parent)
|
||||
};
|
||||
|
||||
if let Some(token) = token_to_revoke {
|
||||
if token != 0 && token != 0xDEAD_BEEF {
|
||||
if let Err(e) = MMU_REVOCATION_QUEUE.push(token) {
|
||||
panic!("FATAL: Критическое переполнение очереди отзыва MMU: {}", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(parent) = parent_idx {
|
||||
let mut parent_slot = self.slots[parent].lock();
|
||||
parent_slot.children.retain(|&c| c != slot_idx);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn get_cap(&self, slot: usize) -> Option<Capability> {
|
||||
self.slots.get(slot).map(|s| s.lock().cap)
|
||||
}
|
||||
|
||||
69
kernel/src/cpu/idt.rs
Normal file
69
kernel/src/cpu/idt.rs
Normal file
@@ -0,0 +1,69 @@
|
||||
use core::arch::asm;
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
#[repr(C, packed)]
|
||||
pub struct IdtEntry {
|
||||
offset_low: u16,
|
||||
selector: u16,
|
||||
ist: u8,
|
||||
type_attr: u8,
|
||||
offset_mid: u16,
|
||||
offset_high: u32,
|
||||
ignore: u32,
|
||||
}
|
||||
|
||||
impl IdtEntry {
|
||||
pub const fn new() -> Self {
|
||||
Self {
|
||||
offset_low: 0,
|
||||
selector: 0,
|
||||
ist: 0,
|
||||
type_attr: 0,
|
||||
offset_mid: 0,
|
||||
offset_high: 0,
|
||||
ignore: 0,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn set_handler(&mut self, handler: u64, selector: u16, flags: u8) {
|
||||
self.offset_low = handler as u16;
|
||||
self.selector = selector;
|
||||
self.ist = 0;
|
||||
self.type_attr = flags | 0x80; // Present bit
|
||||
self.offset_mid = (handler >> 16) as u16;
|
||||
self.offset_high = (handler >> 32) as u32;
|
||||
self.ignore = 0;
|
||||
}
|
||||
}
|
||||
|
||||
#[repr(C, packed)]
|
||||
pub struct IdtPtr {
|
||||
limit: u16,
|
||||
base: u64,
|
||||
}
|
||||
|
||||
pub struct InterruptDescriptorTable {
|
||||
entries: [IdtEntry; 256],
|
||||
}
|
||||
|
||||
impl InterruptDescriptorTable {
|
||||
pub const fn new() -> Self {
|
||||
Self { entries: [IdtEntry::new(); 256] }
|
||||
}
|
||||
|
||||
pub fn set_handler(&mut self, vector: u8, handler: u64) {
|
||||
// 0x8E = Interrupt Gate, Ring 0, Present
|
||||
self.entries[vector as usize].set_handler(handler, 0x28, 0x8E);
|
||||
}
|
||||
|
||||
pub unsafe fn load(&'static self) {
|
||||
let ptr = IdtPtr {
|
||||
limit: (core::mem::size_of::<Self>() - 1) as u16,
|
||||
base: self as *const _ as u64,
|
||||
};
|
||||
unsafe {
|
||||
asm!("lidt [{}]", in(reg) &ptr, options(readonly, nostack, preserves_flags));
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
312
kernel/src/cpu/interrupts.rs
Normal file
312
kernel/src/cpu/interrupts.rs
Normal file
@@ -0,0 +1,312 @@
|
||||
use core::arch::global_asm;
|
||||
use core::arch::asm;
|
||||
use crate::mem::vmm::KERNEL_SPACE;
|
||||
use crate::mem::address::VirtAddr;
|
||||
|
||||
pub static mut IDT: crate::cpu::idt::InterruptDescriptorTable = crate::cpu::idt::InterruptDescriptorTable::new();
|
||||
|
||||
pub const TLB_SHOOTDOWN_VECTOR: u8 = 0xFD;
|
||||
|
||||
macro_rules! exception_stub {
|
||||
($name:ident, $handler:ident) => {
|
||||
concat!(
|
||||
".global ", stringify!($name), "\n",
|
||||
stringify!($name), ":\n",
|
||||
"push rax\npush rcx\npush rdx\npush rbx\npush rbp\n",
|
||||
"push rsi\npush rdi\npush r8\npush r9\npush r10\n",
|
||||
"push r11\npush r12\npush r13\npush r14\npush r15\n",
|
||||
"mov rdi, [rsp + 15*8]\n",
|
||||
"call ", stringify!($handler), "\n",
|
||||
"pop r15\npop r14\npop r13\npop r12\npop r11\n",
|
||||
"pop r10\npop r9\npop r8\npop rdi\npop rsi\n",
|
||||
"pop rbp\npop rbx\npop rdx\npop rcx\npop rax\n",
|
||||
"add rsp, 8\n",
|
||||
"iretq\n",
|
||||
)
|
||||
};
|
||||
}
|
||||
|
||||
global_asm!(
|
||||
exception_stub!(page_fault_stub, rust_page_fault_handler),
|
||||
exception_stub!(gpf_stub, rust_gpf_handler),
|
||||
exception_stub!(double_fault_stub, rust_double_fault_handler),
|
||||
".global tlb_shootdown_stub",
|
||||
"tlb_shootdown_stub:",
|
||||
"push rax",
|
||||
"push rcx",
|
||||
"push rdx",
|
||||
"push rbx",
|
||||
"push rbp",
|
||||
"push rsi",
|
||||
"push rdi",
|
||||
"push r8",
|
||||
"push r9",
|
||||
"push r10",
|
||||
"push r11",
|
||||
"push r12",
|
||||
"push r13",
|
||||
"push r14",
|
||||
"push r15",
|
||||
"call rust_tlb_shootdown_handler",
|
||||
"pop r15",
|
||||
"pop r14",
|
||||
"pop r13",
|
||||
"pop r12",
|
||||
"pop r11",
|
||||
"pop r10",
|
||||
"pop r9",
|
||||
"pop r8",
|
||||
"pop rdi",
|
||||
"pop rsi",
|
||||
"pop rbp",
|
||||
"pop rbx",
|
||||
"pop rdx",
|
||||
"pop rcx",
|
||||
"pop rax",
|
||||
"iretq"
|
||||
);
|
||||
|
||||
unsafe extern "C" {
|
||||
fn page_fault_stub();
|
||||
fn gpf_stub();
|
||||
fn double_fault_stub();
|
||||
fn tlb_shootdown_stub();
|
||||
}
|
||||
|
||||
pub fn init_early_exceptions() {
|
||||
unsafe {
|
||||
let idt = core::ptr::addr_of_mut!(IDT);
|
||||
let early: [u64; 32] = [
|
||||
early_handler_0 as u64, early_handler_1 as u64,
|
||||
early_handler_2 as u64, early_handler_3 as u64,
|
||||
early_handler_4 as u64, early_handler_5 as u64,
|
||||
early_handler_6 as u64, early_handler_7 as u64,
|
||||
early_handler_8 as u64, early_handler_9 as u64,
|
||||
early_handler_10 as u64, early_handler_11 as u64,
|
||||
early_handler_12 as u64, early_handler_13 as u64,
|
||||
early_handler_14 as u64, early_handler_15 as u64,
|
||||
early_handler_16 as u64, early_handler_17 as u64,
|
||||
early_handler_18 as u64, early_handler_19 as u64,
|
||||
early_handler_20 as u64, early_handler_21 as u64,
|
||||
early_handler_22 as u64, early_handler_23 as u64,
|
||||
early_handler_24 as u64, early_handler_25 as u64,
|
||||
early_handler_26 as u64, early_handler_27 as u64,
|
||||
early_handler_28 as u64, early_handler_29 as u64,
|
||||
early_handler_30 as u64, early_handler_31 as u64,
|
||||
];
|
||||
for (v, &handler) in early.iter().enumerate() {
|
||||
(*idt).set_handler(v as u8, handler);
|
||||
}
|
||||
// Override with proper handlers for vectors 8, 13, 14 and TLB IPI.
|
||||
(*idt).set_handler(8, double_fault_stub as u64);
|
||||
(*idt).set_handler(13, gpf_stub as u64);
|
||||
(*idt).set_handler(14, page_fault_stub as u64);
|
||||
(*idt).set_handler(TLB_SHOOTDOWN_VECTOR, tlb_shootdown_stub as u64);
|
||||
|
||||
let ptr: &'static crate::cpu::idt::InterruptDescriptorTable = &*core::ptr::addr_of!(IDT);
|
||||
ptr.load();
|
||||
}
|
||||
}
|
||||
|
||||
pub fn init_idt() {
|
||||
// init_early_exceptions already loaded the IDT; this just overrides
|
||||
// vectors that the full kernel needs. Calling lidt again is harmless.
|
||||
unsafe {
|
||||
let idt = core::ptr::addr_of_mut!(IDT);
|
||||
(*idt).set_handler(14, page_fault_stub as u64);
|
||||
(*idt).set_handler(TLB_SHOOTDOWN_VECTOR, tlb_shootdown_stub as u64);
|
||||
let ptr: &'static crate::cpu::idt::InterruptDescriptorTable = &*core::ptr::addr_of!(IDT);
|
||||
ptr.load();
|
||||
}
|
||||
}
|
||||
|
||||
#[unsafe(no_mangle)]
|
||||
pub extern "C" fn rust_gpf_handler(error_code: u64) -> ! {
|
||||
crate::debug::serial::write_global(format_args!(
|
||||
"\n!!! GENERAL PROTECTION FAULT !!! error_code={:#x}\n\
|
||||
CPU halted.\n",
|
||||
error_code
|
||||
));
|
||||
loop {
|
||||
unsafe { asm!("cli; hlt", options(nomem, nostack, preserves_flags)); }
|
||||
}
|
||||
}
|
||||
|
||||
#[unsafe(no_mangle)]
|
||||
pub extern "C" fn rust_double_fault_handler(error_code: u64) -> ! {
|
||||
crate::debug::serial::write_global(format_args!(
|
||||
"\n!!! DOUBLE FAULT !!! error_code={:#x}\n\
|
||||
CPU halted.\n",
|
||||
error_code
|
||||
));
|
||||
loop {
|
||||
unsafe { asm!("cli; hlt", options(nomem, nostack, preserves_flags)); }
|
||||
}
|
||||
}
|
||||
|
||||
global_asm!(r#"
|
||||
.altmacro
|
||||
|
||||
.macro early_stub vec
|
||||
.globl early_handler_\vec
|
||||
.balign 16
|
||||
early_handler_\vec:
|
||||
push 0 /* dummy error code */
|
||||
push \vec /* vector number */
|
||||
jmp early_common
|
||||
.endm
|
||||
|
||||
early_stub 0
|
||||
early_stub 1
|
||||
early_stub 2
|
||||
early_stub 3
|
||||
early_stub 4
|
||||
early_stub 5
|
||||
early_stub 6
|
||||
early_stub 7
|
||||
early_stub 8
|
||||
early_stub 9
|
||||
early_stub 10
|
||||
early_stub 11
|
||||
early_stub 12
|
||||
early_stub 13
|
||||
early_stub 14
|
||||
early_stub 15
|
||||
early_stub 16
|
||||
early_stub 17
|
||||
early_stub 18
|
||||
early_stub 19
|
||||
early_stub 20
|
||||
early_stub 21
|
||||
early_stub 22
|
||||
early_stub 23
|
||||
early_stub 24
|
||||
early_stub 25
|
||||
early_stub 26
|
||||
early_stub 27
|
||||
early_stub 28
|
||||
early_stub 29
|
||||
early_stub 30
|
||||
early_stub 31
|
||||
|
||||
early_common:
|
||||
push rax
|
||||
push rcx
|
||||
push rdx
|
||||
push rbx
|
||||
push rbp
|
||||
push rsi
|
||||
push rdi
|
||||
push r8
|
||||
push r9
|
||||
push r10
|
||||
push r11
|
||||
push r12
|
||||
push r13
|
||||
push r14
|
||||
push r15
|
||||
mov rdi, [rsp + 15*8] /* vector number */
|
||||
mov rsi, [rsp + 16*8] /* error code (or dummy 0) */
|
||||
call rust_early_exception_handler
|
||||
/* never returns */
|
||||
"#);
|
||||
|
||||
unsafe extern "C" {
|
||||
fn early_handler_0();
|
||||
fn early_handler_1();
|
||||
fn early_handler_2();
|
||||
fn early_handler_3();
|
||||
fn early_handler_4();
|
||||
fn early_handler_5();
|
||||
fn early_handler_6();
|
||||
fn early_handler_7();
|
||||
fn early_handler_8();
|
||||
fn early_handler_9();
|
||||
fn early_handler_10();
|
||||
fn early_handler_11();
|
||||
fn early_handler_12();
|
||||
fn early_handler_13();
|
||||
fn early_handler_14();
|
||||
fn early_handler_15();
|
||||
fn early_handler_16();
|
||||
fn early_handler_17();
|
||||
fn early_handler_18();
|
||||
fn early_handler_19();
|
||||
fn early_handler_20();
|
||||
fn early_handler_21();
|
||||
fn early_handler_22();
|
||||
fn early_handler_23();
|
||||
fn early_handler_24();
|
||||
fn early_handler_25();
|
||||
fn early_handler_26();
|
||||
fn early_handler_27();
|
||||
fn early_handler_28();
|
||||
fn early_handler_29();
|
||||
fn early_handler_30();
|
||||
fn early_handler_31();
|
||||
}
|
||||
|
||||
#[unsafe(no_mangle)]
|
||||
pub extern "C" fn rust_early_exception_handler(vector: u64, _error_code: u64) -> ! {
|
||||
crate::debug::serial::write_global(format_args!(
|
||||
"\n!!! EARLY EXCEPTION !!! vector={} error_code={:#x}\n\
|
||||
CPU halted.\n",
|
||||
vector, _error_code
|
||||
));
|
||||
loop {
|
||||
unsafe { asm!("cli; hlt", options(nomem, nostack, preserves_flags)); }
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
#[unsafe(no_mangle)]
|
||||
pub extern "C" fn rust_page_fault_handler(error_code: u64) {
|
||||
let fault_addr: u64;
|
||||
unsafe {
|
||||
asm!("mov {}, cr2", out(reg) fault_addr, options(nomem, nostack, preserves_flags));
|
||||
}
|
||||
|
||||
let write = (error_code & 0x2) != 0;
|
||||
let present = (error_code & 0x1) != 0;
|
||||
let virt_addr = VirtAddr(fault_addr);
|
||||
|
||||
// Acquire KERNEL_SPACE lock once for both deferred events and fault handling.
|
||||
// TODO SMP: switch to RwLock so concurrent read-only faults are not
|
||||
// serialised. `handle_fault` is read-only for COW faults but mutates
|
||||
// the VMA tree on demand-paging — a lock-free VMA tree or per-region
|
||||
// locks would be ideal.
|
||||
let mut vmm_guard = KERNEL_SPACE.lock();
|
||||
|
||||
if let Some(space) = vmm_guard.as_mut() {
|
||||
space.process_pending_revocations();
|
||||
|
||||
match space.handle_fault(virt_addr, write) {
|
||||
Ok(_) => {},
|
||||
Err(e) => {
|
||||
panic!(
|
||||
"KERNEL PANIC: Unprocessed failure of virtual memory (Page Fault)!\n\
|
||||
Address: {:#X}\n\
|
||||
Mode: {}\n\
|
||||
Presence: {}\n\
|
||||
Cause VMM: {:?}",
|
||||
fault_addr,
|
||||
if write { "WRITE" } else { "READ" },
|
||||
if present { "YES (Protection Violation)" } else { "NO (Not Present)" },
|
||||
e
|
||||
);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
panic!(
|
||||
"KERNEL PANIC: Critical Page Fault before alocate global KERNEL_SPACE!\n\
|
||||
Адрес сбоя: {:#X}",
|
||||
fault_addr
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[unsafe(no_mangle)]
|
||||
pub extern "C" fn rust_tlb_shootdown_handler() {
|
||||
crate::mem::vmm::handle_tlb_shootdown_ipi();
|
||||
crate::cpu::lapic::send_eoi();
|
||||
}
|
||||
51
kernel/src/cpu/lapic.rs
Normal file
51
kernel/src/cpu/lapic.rs
Normal file
@@ -0,0 +1,51 @@
|
||||
use core::sync::atomic::{AtomicU64, Ordering};
|
||||
use crate::mem::address::get_hhdm;
|
||||
|
||||
pub const LAPIC_DEFAULT_BASE: u64 = 0xFEE00_000;
|
||||
const LAPIC_EOI: u64 = 0x0B0;
|
||||
const LAPIC_ICR_LOW: u64 = 0x300;
|
||||
|
||||
static LAPIC_VIRT_BASE: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
pub fn init() {
|
||||
LAPIC_VIRT_BASE.store(LAPIC_DEFAULT_BASE + get_hhdm(), Ordering::SeqCst);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn current_core_id() -> u32 {
|
||||
let mut ebx: u32;
|
||||
unsafe {
|
||||
core::arch::asm!(
|
||||
"mov {tmp:r}, rbx",
|
||||
"mov eax, 1",
|
||||
"cpuid",
|
||||
"mov {out:e}, ebx",
|
||||
"mov rbx, {tmp:r}",
|
||||
tmp = out(reg) _,
|
||||
out = out(reg) ebx,
|
||||
out("eax") _, out("ecx") _, out("edx") _,
|
||||
options(nostack, preserves_flags)
|
||||
);
|
||||
}
|
||||
ebx >> 24
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn write_lapic_reg(offset: u64, value: u32) {
|
||||
let base = LAPIC_VIRT_BASE.load(Ordering::Relaxed);
|
||||
if base == 0 { return; }
|
||||
unsafe { core::ptr::write_volatile((base + offset) as *mut u32, value) }
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn send_eoi() {
|
||||
write_lapic_reg(LAPIC_EOI, 0);
|
||||
}
|
||||
|
||||
pub fn broadcast_ipi_exclude_self(vector: u8) {
|
||||
// Destination Shorthand = 10b (All excluding self)
|
||||
// Level = 1 (Assert)
|
||||
// Delivery Mode = 000b
|
||||
let icr_low = (2 << 18) | (1 << 14) | (vector as u32);
|
||||
write_lapic_reg(LAPIC_ICR_LOW, icr_low);
|
||||
}
|
||||
4
kernel/src/cpu/mod.rs
Normal file
4
kernel/src/cpu/mod.rs
Normal file
@@ -0,0 +1,4 @@
|
||||
pub mod idt;
|
||||
pub mod interrupts;
|
||||
pub mod lapic;
|
||||
pub mod smp;
|
||||
61
kernel/src/cpu/smp.rs
Normal file
61
kernel/src/cpu/smp.rs
Normal file
@@ -0,0 +1,61 @@
|
||||
use core::sync::atomic::{AtomicU64, Ordering};
|
||||
use limine::mp;
|
||||
use crate::mem::vmm::ACTIVE_CPUS_MASK;
|
||||
|
||||
static KERNEL_PML4: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
pub fn set_kernel_pml4(phys: u64) {
|
||||
KERNEL_PML4.store(phys, Ordering::Release);
|
||||
}
|
||||
|
||||
pub fn start_aps(mp_res: &limine::response::MpResponse) {
|
||||
let bsp_lapic = mp_res.bsp_lapic_id();
|
||||
|
||||
let mut ap_count = 0;
|
||||
for cpu in mp_res.cpus() {
|
||||
if cpu.lapic_id == bsp_lapic {
|
||||
continue;
|
||||
}
|
||||
cpu.goto_address.write(ap_entry);
|
||||
ap_count += 1;
|
||||
}
|
||||
|
||||
if ap_count == 0 {
|
||||
return;
|
||||
}
|
||||
|
||||
let target_mask: u64 = mp_res.cpus().iter()
|
||||
.filter(|cpu| cpu.lapic_id != bsp_lapic && (cpu.lapic_id as u64) < 64)
|
||||
.fold(0, |mask, cpu| mask | (1u64 << cpu.lapic_id));
|
||||
|
||||
while ACTIVE_CPUS_MASK.load(Ordering::Acquire) & target_mask != target_mask {
|
||||
core::hint::spin_loop();
|
||||
}
|
||||
}
|
||||
|
||||
extern "C" fn ap_entry(cpu: &mp::Cpu) -> ! {
|
||||
let pml4 = KERNEL_PML4.load(Ordering::Acquire);
|
||||
if pml4 != 0 {
|
||||
unsafe {
|
||||
core::arch::asm!("mov cr3, {0}", in(reg) pml4,
|
||||
options(nostack, preserves_flags));
|
||||
}
|
||||
}
|
||||
|
||||
unsafe {
|
||||
(*core::ptr::addr_of!(crate::cpu::interrupts::IDT)).load();
|
||||
}
|
||||
|
||||
crate::cpu::lapic::init();
|
||||
|
||||
let lapic_id = cpu.lapic_id;
|
||||
if (lapic_id as u64) < 64 {
|
||||
ACTIVE_CPUS_MASK.fetch_or(1u64 << lapic_id, Ordering::AcqRel);
|
||||
}
|
||||
|
||||
unsafe { core::arch::asm!("sti", options(nomem, nostack, preserves_flags)); }
|
||||
|
||||
loop {
|
||||
unsafe { core::arch::asm!("hlt", options(nomem, nostack, preserves_flags)); }
|
||||
}
|
||||
}
|
||||
@@ -1,8 +1,5 @@
|
||||
pub mod serial;
|
||||
|
||||
use embedded_graphics::pixelcolor::Rgb888;
|
||||
use embedded_graphics::prelude::RgbColor;
|
||||
|
||||
pub enum LogLevel {
|
||||
Info, Warn, Error,
|
||||
}
|
||||
@@ -16,11 +13,11 @@ impl LogLevel {
|
||||
}
|
||||
}
|
||||
|
||||
pub fn console_color(&self) -> Rgb888 {
|
||||
pub fn console_color(&self) -> u32 {
|
||||
match self {
|
||||
LogLevel::Info => Rgb888::GREEN,
|
||||
LogLevel::Warn => Rgb888::YELLOW,
|
||||
LogLevel::Error => Rgb888::RED,
|
||||
LogLevel::Info => 0x00FF00, // GREEN
|
||||
LogLevel::Warn => 0xFFFF00, // YELLOW
|
||||
LogLevel::Error => 0xFF0000, // RED
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -29,26 +26,23 @@ impl LogLevel {
|
||||
macro_rules! log {
|
||||
($console:expr, $level:expr, $module:expr, $($arg:tt)*) => {{
|
||||
use core::fmt::Write;
|
||||
use embedded_graphics::pixelcolor::Rgb888;
|
||||
|
||||
// Visual screen output
|
||||
$console.set_color($level.console_color());
|
||||
let _ = write!($console, "[ LOG ] ");
|
||||
|
||||
$console.set_color(Rgb888::WHITE);
|
||||
$console.set_color(0xFFFFFF); // WHITE
|
||||
let _ = write!($console, "{:<6} | ", $module);
|
||||
let _ = writeln!($console, $($arg)*);
|
||||
|
||||
// Serial debug output
|
||||
let mut sp = unsafe { $crate::debug::serial::SerialPort::init() };
|
||||
let _ = writeln!(
|
||||
sp,
|
||||
"{}[{:>5}]\x1b[0m {:<8} | {}",
|
||||
$crate::debug::serial::write_global(format_args!(
|
||||
"{}[{:>5}]\x1b[0m {:<8} | {}\n",
|
||||
$level.serial_color_code(),
|
||||
"LOG",
|
||||
$module,
|
||||
format_args!($($arg)*)
|
||||
);
|
||||
));
|
||||
}};
|
||||
}
|
||||
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
use core::arch::asm;
|
||||
use core::fmt::Write;
|
||||
use crate::mem::allocator::Locked;
|
||||
|
||||
pub struct SerialPort(u16);
|
||||
|
||||
@@ -7,16 +9,14 @@ impl SerialPort {
|
||||
|
||||
pub unsafe fn init() -> Self {
|
||||
let port = Self::COM1;
|
||||
// В новых версиях Rust даже внутри unsafe fn
|
||||
// вызовы других unsafe функций требуют явного блока
|
||||
unsafe {
|
||||
outb(port + 1, 0x00); // Disable interrupts
|
||||
outb(port + 3, 0x80); // Enable DLAB
|
||||
outb(port + 0, 0x03); // Divisor 3 (38400 baud)
|
||||
outb(port + 1, 0x00);
|
||||
outb(port + 3, 0x03); // 8 bits, no parity, 1 stop bit
|
||||
outb(port + 2, 0xC7); // Enable FIFO
|
||||
outb(port + 4, 0x0B); // IRQs enabled, RTS/DSR set
|
||||
outb(port + 3, 0x80);
|
||||
outb(port + 0, 0x03);
|
||||
outb(port + 1, 0x00);
|
||||
outb(port + 3, 0x03);
|
||||
outb(port + 2, 0xC7);
|
||||
outb(port + 4, 0x0B);
|
||||
}
|
||||
SerialPort(port)
|
||||
}
|
||||
@@ -38,6 +38,20 @@ impl core::fmt::Write for SerialPort {
|
||||
}
|
||||
}
|
||||
|
||||
static SERIAL_PORT: Locked<Option<SerialPort>> = Locked::new(None);
|
||||
|
||||
pub fn init_global() {
|
||||
let mut guard = SERIAL_PORT.lock();
|
||||
*guard = Some(unsafe { SerialPort::init() });
|
||||
}
|
||||
|
||||
pub fn write_global(args: core::fmt::Arguments) {
|
||||
let mut guard = SERIAL_PORT.lock();
|
||||
if let Some(ref mut sp) = *guard {
|
||||
let _ = sp.write_fmt(args);
|
||||
}
|
||||
}
|
||||
|
||||
unsafe fn outb(port: u16, val: u8) {
|
||||
unsafe {
|
||||
asm!("out dx, al", in("dx") port, in("al") val, options(nomem, nostack, preserves_flags));
|
||||
|
||||
67
kernel/src/events.rs
Normal file
67
kernel/src/events.rs
Normal file
@@ -0,0 +1,67 @@
|
||||
use core::sync::atomic::{AtomicU64, AtomicUsize, Ordering};
|
||||
|
||||
const QUEUE_SIZE: usize = 1024;
|
||||
const QUEUE_MASK: usize = QUEUE_SIZE - 1;
|
||||
|
||||
pub struct RevocationQueue {
|
||||
buffer: [AtomicU64; QUEUE_SIZE],
|
||||
#[doc(hidden)]
|
||||
_pad0: [u8; 64],
|
||||
head: AtomicUsize,
|
||||
#[doc(hidden)]
|
||||
_pad1: [u8; 64],
|
||||
tail: AtomicUsize,
|
||||
}
|
||||
|
||||
impl RevocationQueue {
|
||||
pub const fn new() -> Self {
|
||||
Self {
|
||||
#[allow(clippy::declare_interior_mutable_const)]
|
||||
buffer: [const { AtomicU64::new(0) }; QUEUE_SIZE],
|
||||
_pad0: [0; 64],
|
||||
head: AtomicUsize::new(0),
|
||||
_pad1: [0; 64],
|
||||
tail: AtomicUsize::new(0),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn push(&self, token_sig: u64) -> Result<(), &'static str> {
|
||||
let mut tail = self.tail.load(Ordering::Relaxed);
|
||||
loop {
|
||||
let head = self.head.load(Ordering::Acquire);
|
||||
|
||||
if tail.wrapping_sub(head) >= QUEUE_SIZE {
|
||||
return Err("Revocation queue overflow");
|
||||
}
|
||||
|
||||
match self.tail.compare_exchange_weak(
|
||||
tail,
|
||||
tail.wrapping_add(1),
|
||||
Ordering::AcqRel,
|
||||
Ordering::Relaxed
|
||||
) {
|
||||
Ok(_) => {
|
||||
self.buffer[tail & QUEUE_MASK].store(token_sig, Ordering::Release);
|
||||
return Ok(());
|
||||
}
|
||||
Err(actual_tail) => tail = actual_tail,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn pop(&self) -> Option<u64> {
|
||||
let head = self.head.load(Ordering::Relaxed);
|
||||
let tail = self.tail.load(Ordering::Acquire);
|
||||
|
||||
if head == tail {
|
||||
return None;
|
||||
}
|
||||
|
||||
let token = self.buffer[head & QUEUE_MASK].load(Ordering::Acquire);
|
||||
self.head.store(head.wrapping_add(1), Ordering::Release);
|
||||
|
||||
Some(token)
|
||||
}
|
||||
}
|
||||
|
||||
pub static MMU_REVOCATION_QUEUE: RevocationQueue = RevocationQueue::new();
|
||||
BIN
kernel/src/font.psf
Normal file
BIN
kernel/src/font.psf
Normal file
Binary file not shown.
@@ -8,14 +8,23 @@ use core::fmt::{self, Write};
|
||||
use alloc::vec::Vec;
|
||||
|
||||
use limine::BaseRevision;
|
||||
use limine::request::{FramebufferRequest, RequestsEndMarker, RequestsStartMarker, HhdmRequest, MemoryMapRequest, ExecutableAddressRequest};
|
||||
use limine::request::{FramebufferRequest, RequestsEndMarker, RequestsStartMarker, HhdmRequest, MemoryMapRequest, ExecutableAddressRequest, MpRequest};
|
||||
|
||||
use crate::mem::paging::{PageTable, PageTableFlags};
|
||||
use crate::mem::address::{PhysAddr, VirtAddr};
|
||||
use crate::cap::{Relation, CapRights, Capability, CapObject};
|
||||
use crate::mem::pm_manages::{PMActor, PMRequest, PMResult};
|
||||
use crate::mem::buddy::BuddyAllocator;
|
||||
|
||||
pub mod cap;
|
||||
pub mod cpu;
|
||||
pub mod events;
|
||||
pub mod tty;
|
||||
|
||||
mod mem;
|
||||
|
||||
static KERNEL_FONT: &[u8] = include_bytes!("font.psf");
|
||||
|
||||
#[macro_use]
|
||||
pub mod debug;
|
||||
|
||||
@@ -152,6 +161,9 @@ static BASE_REVISION: BaseRevision = BaseRevision::new();
|
||||
#[used] #[unsafe(link_section = ".requests")]
|
||||
static FRAMEBUFFER_REQUEST: FramebufferRequest = FramebufferRequest::new();
|
||||
|
||||
#[used] #[unsafe(link_section = ".requests")]
|
||||
static MP_REQUEST: MpRequest = MpRequest::new();
|
||||
|
||||
#[used] #[unsafe(link_section = ".requests_start_marker")]
|
||||
static _START_MARKER: RequestsStartMarker = RequestsStartMarker::new();
|
||||
|
||||
@@ -167,52 +179,95 @@ unsafe extern "C" fn kmain() -> ! {
|
||||
let hhdm_res = HHDM_REQUEST.get_response().expect("Limine: No HHDM");
|
||||
let kaddr_res = KERNEL_ADDR_REQUEST.get_response().expect("Limine: No Kernel Address");
|
||||
let hhdm_offset = hhdm_res.offset();
|
||||
crate::mem::address::init_hhdm(hhdm_offset);
|
||||
|
||||
let fb = fb_res.framebuffers().next().expect("Limine: No active framebuffer found");
|
||||
let mut console = Console::new(FramebufferDisplay { framebuffer: &fb });
|
||||
let mut console = tty::Console::new(&fb, KERNEL_FONT);
|
||||
console.clear();
|
||||
|
||||
info!(console, "BOOT", "LISA Kernel Starting...");
|
||||
debug::serial::init_global();
|
||||
|
||||
unsafe { mem::pmm::BitmapPMM::init(&mmap_res, hhdm_offset); }
|
||||
info!(console, "MEM", "Physical Memory Manager initialized.");
|
||||
cpu::interrupts::init_early_exceptions();
|
||||
info!(console, "BOOT", "LIS4 Kernel Starting...");
|
||||
|
||||
unsafe { mem::pmm::BitmapPMM::init(&mmap_res); }
|
||||
info!(console, "MEM", "Primary Physical Memory Manager (BitmapPMM) initialized.");
|
||||
|
||||
cpu::lapic::init();
|
||||
info!(console, "LAPIC", "Local APIC initialized.");
|
||||
|
||||
// debug: locate the free page
|
||||
info!(console, "BOOT", "alloc_frame...");
|
||||
let p4_phys = mem::pmm::alloc_frame().expect("OOM: Failed to allocate P4 table");
|
||||
let p4 = unsafe { &mut *p4_phys.to_virt(hhdm_offset).as_mut_ptr::<PageTable>() };
|
||||
info!(console, "BOOT", "alloc_frame ok: phys=0x{:x}", p4_phys.0);
|
||||
let virt = p4_phys.to_virt();
|
||||
info!(console, "BOOT", "virt=0x{:x}", virt.0);
|
||||
let p4 = unsafe { &mut *virt.as_mut_ptr::<PageTable>() };
|
||||
info!(console, "BOOT", "zeroing page...");
|
||||
unsafe { core::ptr::write_bytes(p4 as *mut _ as *mut u8, 0, 4096); }
|
||||
info!(console, "BOOT", "zero done");
|
||||
|
||||
let flags = PageTableFlags::PRESENT | PageTableFlags::WRITABLE;
|
||||
|
||||
for entry in mmap_res.entries() {
|
||||
for (i, entry) in mmap_res.entries().iter().enumerate() {
|
||||
let phys = PhysAddr(entry.base);
|
||||
let virt_hhdm = phys.to_virt(hhdm_offset);
|
||||
p4.map_region(virt_hhdm, phys, entry.length, flags, hhdm_offset);
|
||||
let virt_hhdm = phys.to_virt();
|
||||
p4.map_region(virt_hhdm, phys, entry.length, flags);
|
||||
if entry.entry_type != limine::memory_map::EntryType::RESERVED {
|
||||
p4.map_region(VirtAddr(entry.base), phys, entry.length, flags, hhdm_offset);
|
||||
p4.map_region(VirtAddr(entry.base), phys, entry.length, flags);
|
||||
}
|
||||
}
|
||||
|
||||
p4.map_region(VirtAddr(kaddr_res.virtual_base()), PhysAddr(kaddr_res.physical_base()), 0x1000 * 1024, flags, hhdm_offset);
|
||||
p4.map_region(
|
||||
VirtAddr(kaddr_res.virtual_base()),
|
||||
PhysAddr(kaddr_res.physical_base()),
|
||||
0x1000 * 1024,
|
||||
flags,
|
||||
);
|
||||
|
||||
info!(console, "MMU", "Switching to Kernel Page Tables...");
|
||||
info!(console, "MMU", "Activating Kernel Page Tables...");
|
||||
unsafe { p4.activate(p4_phys); }
|
||||
cpu::smp::set_kernel_pml4(p4_phys.0);
|
||||
|
||||
let heap_start = 0xFFFF_9000_0000_0000;
|
||||
let heap_size = 8 * 1024 * 1024;
|
||||
for i in (0..heap_size).step_by(4096) {
|
||||
let frame = mem::pmm::alloc_frame().expect("OOM: Heap allocation failed");
|
||||
p4.map_page(VirtAddr(heap_start + i as u64), frame, flags, hhdm_offset);
|
||||
p4.map_page(VirtAddr(heap_start + i as u64), frame, flags);
|
||||
}
|
||||
|
||||
{
|
||||
let mut allocator = allocator::ALLOCATOR.lock();
|
||||
allocator.init(heap_start as usize, heap_size);
|
||||
}
|
||||
info!(console, "HEAP", "Global Allocator is online.");
|
||||
info!(console, "HEAP", "Kernel Slab Allocator is online.");
|
||||
|
||||
mem::init_cpu_features();
|
||||
mem::vmm::init_kernel_space(p4_phys);
|
||||
info!(console, "VMM", "Kernel Address Space registered successfully.");
|
||||
|
||||
cpu::interrupts::init_idt();
|
||||
info!(console, "CPU", "Interrupt Descriptor Table (IDT) loaded.");
|
||||
|
||||
mem::pm_router::init();
|
||||
info!(console, "PM", "PMRouter online. 65536 lock-free routing channels allocated.");
|
||||
|
||||
unsafe { core::arch::asm!("sti", options(nomem, nostack, preserves_flags)); }
|
||||
|
||||
if let Some(mp_res) = MP_REQUEST.get_response() {
|
||||
let cpu_ct = mp_res.cpus().len();
|
||||
let bsp_id = mp_res.bsp_lapic_id();
|
||||
info!(console, "SMP", "Detected {} CPU(s), BSP APIC ID: {}", cpu_ct, bsp_id);
|
||||
if cpu_ct > 1 {
|
||||
cpu::smp::start_aps(mp_res);
|
||||
let active = crate::mem::vmm::ACTIVE_CPUS_MASK.load(core::sync::atomic::Ordering::Relaxed);
|
||||
info!(console, "SMP", "All APs online. Active mask: {:#018x}", active);
|
||||
}
|
||||
} else {
|
||||
info!(console, "SMP", "No MP response — running single-core");
|
||||
}
|
||||
|
||||
let root_cnode = cap::CNode::new(256);
|
||||
|
||||
if let Some(frame) = mem::pmm::alloc_frame() {
|
||||
let mem_cap = Capability {
|
||||
object: CapObject::Memory { phys: frame, size_pages: 1 },
|
||||
@@ -220,23 +275,182 @@ unsafe extern "C" fn kmain() -> ! {
|
||||
relation: Relation::Strong,
|
||||
token_sig: 0x1,
|
||||
};
|
||||
|
||||
root_cnode.insert(0, mem_cap).unwrap();
|
||||
info!(console, "CAP", "Root capability created at slot 0");
|
||||
}
|
||||
|
||||
root_cnode.mint(0, 10, Relation::Borrow, CapRights::READ | CapRights::WRITE).unwrap();
|
||||
|
||||
if let Some(c) = root_cnode.get_cap(10) {
|
||||
info!(console, "CAP", "Slot 10 (Borrowed): {:?}", c.relation);
|
||||
}
|
||||
|
||||
root_cnode.revoke(0);
|
||||
if let Some(c) = root_cnode.get_cap(10) {
|
||||
if !c.is_valid() {
|
||||
info!(console, "CAP", "Slot 10 successfully revoked.");
|
||||
info!(console, "CAP", "Capability ownership and metadata verification systems passed.");
|
||||
|
||||
info!(console, "PM", "=-= PMActor & Buddy Allocator Production Test =-=");
|
||||
|
||||
let total_test_pages = 2048;
|
||||
let actor_base_phys = PhysAddr(0x4000_0000);
|
||||
|
||||
let actor_root_cap = Capability {
|
||||
object: CapObject::Memory { phys: actor_base_phys, size_pages: total_test_pages },
|
||||
rights: CapRights::all(),
|
||||
relation: Relation::Strong,
|
||||
token_sig: 0xAAAA_BBBB,
|
||||
};
|
||||
|
||||
let mut pm_actor = PMActor::new(1, actor_root_cap, actor_base_phys, (total_test_pages as u64) * 4096);
|
||||
info!(console, "PM", "PMActor ID:1 initialized. Initial free pool: {} pages.", pm_actor.free_pages());
|
||||
|
||||
info!(console, "PM", "-> Step 1: Performing asynchronous parallel allocations via PMRouter...");
|
||||
|
||||
let router = mem::pm_router::get_router();
|
||||
|
||||
let ch1 = router.alloc_channel().expect("Router overflow");
|
||||
let ch2 = router.alloc_channel().expect("Router overflow");
|
||||
let ch3 = router.alloc_channel().expect("Router overflow");
|
||||
|
||||
pm_actor.submit_request(PMRequest::Allocate { size_pages: 1, token_sig: 0x11, channel_id: ch1 }).unwrap();
|
||||
pm_actor.submit_request(PMRequest::Allocate { size_pages: 15, token_sig: 0x22, channel_id: ch2 }).unwrap();
|
||||
pm_actor.submit_request(PMRequest::Allocate { size_pages: 256, token_sig: 0x33, channel_id: ch3 }).unwrap();
|
||||
|
||||
let responses1 = pm_actor.process_messages();
|
||||
mem::pm_router::dispatch(responses1);
|
||||
|
||||
let res1 = router.wait_for_response(ch1);
|
||||
let res2 = router.wait_for_response(ch2);
|
||||
let res3 = router.wait_for_response(ch3);
|
||||
|
||||
let mut allocated_caps = Vec::new();
|
||||
|
||||
if let PMResult::Allocated { cap, order } = res1 {
|
||||
let phys_addr = if let CapObject::Memory { phys, .. } = cap.object { phys.0 } else { 0 };
|
||||
info!(console, "PM", " [OK] ch:{} | Allocated 1 page (Order {}) @ {:#X}", ch1, order, phys_addr);
|
||||
allocated_caps.push((cap, order));
|
||||
}
|
||||
if let PMResult::Allocated { cap, order } = res2 {
|
||||
let phys_addr = if let CapObject::Memory { phys, .. } = cap.object { phys.0 } else { 0 };
|
||||
info!(console, "PM", " [OK] ch:{} | Allocated 15 pages (Order {}) @ {:#X}", ch2, order, phys_addr);
|
||||
allocated_caps.push((cap, order));
|
||||
}
|
||||
if let PMResult::Allocated { cap, order } = res3 {
|
||||
let phys_addr = if let CapObject::Memory { phys, .. } = cap.object { phys.0 } else { 0 };
|
||||
info!(console, "PM", " [OK] ch:{} | Allocated 256 pages (Order {}) @ {:#X}", ch3, order, phys_addr);
|
||||
allocated_caps.push((cap, order));
|
||||
}
|
||||
|
||||
info!(console, "PM", " Free space after allocations: {} / {} pages.", pm_actor.free_pages(), total_test_pages);
|
||||
|
||||
info!(console, "PM", "-> Step 2: Testing Out-Of-Memory interception...");
|
||||
|
||||
let ch_oom = router.alloc_channel().expect("Router overflow");
|
||||
pm_actor.submit_request(PMRequest::Allocate { size_pages: 4096, token_sig: 0xDEAD, channel_id: ch_oom }).unwrap();
|
||||
|
||||
let responses_oom = pm_actor.process_messages();
|
||||
mem::pm_router::dispatch(responses_oom);
|
||||
|
||||
if let PMResult::OutOfMemory { size_pages } = router.wait_for_response(ch_oom) {
|
||||
info!(console, "PM", " [OK] OOM condition correctly handled for request of {} pages.", size_pages);
|
||||
} else {
|
||||
panic!("PM: Failed OOM validation!");
|
||||
}
|
||||
|
||||
info!(console, "PM", "-> Step 3: Verifying static sub-region Carving allocation...");
|
||||
|
||||
let ch_carve = router.alloc_channel().expect("Router overflow");
|
||||
pm_actor.submit_request(PMRequest::Carve { offset_pages: 100, size_pages: 10, channel_id: ch_carve }).unwrap();
|
||||
|
||||
let responses_carve = pm_actor.process_messages();
|
||||
mem::pm_router::dispatch(responses_carve);
|
||||
|
||||
if let PMResult::Carved { cap } = router.wait_for_response(ch_carve) {
|
||||
let phys_addr = if let CapObject::Memory { phys, .. } = cap.object { phys.0 } else { 0 };
|
||||
info!(console, "PM", " [OK] Fixed sub-region carved at absolute address: {:#X}", phys_addr);
|
||||
} else {
|
||||
panic!("PM: Failed Carve validation!");
|
||||
}
|
||||
|
||||
info!(console, "PM", "-> Step 4: Submitting Free requests & evaluating Buddy coalescence...");
|
||||
|
||||
for (cap, order) in allocated_caps {
|
||||
if let CapObject::Memory { phys, .. } = cap.object {
|
||||
let rel_idx = ((phys.0 - actor_base_phys.0) / 4096) as usize;
|
||||
pm_actor.submit_request(PMRequest::Free { local_frame_idx: rel_idx, order }).unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
let responses_free = pm_actor.process_messages();
|
||||
mem::pm_router::dispatch(responses_free);
|
||||
|
||||
let final_free = pm_actor.free_pages();
|
||||
info!(console, "PM", " All test components recycled. Total free memory: {} pages.", final_free);
|
||||
|
||||
if final_free == total_test_pages {
|
||||
info!(console, "PM", "~) ALL ACTOR/ROUTER SYSTEM TESTS PASSED SUCCESSFULLY (~ :3");
|
||||
} else {
|
||||
panic!("CRITICAL STATE LOSS: Memory leak detected inside PMActor context!");
|
||||
}
|
||||
|
||||
info!(console, "PM", "=-= Buddy Allocator Direct Test (Intrusive List) =-=");
|
||||
|
||||
let buddy_test_pages = 128;
|
||||
let buddy_test_base = PhysAddr(0x5000_0000);
|
||||
let mut buddy = BuddyAllocator::new(buddy_test_pages, buddy_test_base.0);
|
||||
|
||||
let b1 = buddy.alloc(0).expect("buddy: alloc order 0");
|
||||
let b2 = buddy.alloc(0).expect("buddy: alloc order 0 #2");
|
||||
buddy.free(b1, 0);
|
||||
buddy.free(b2, 0);
|
||||
assert_eq!(buddy.free_pages(), buddy_test_pages, "buddy: pages not fully recovered after simple alloc/free");
|
||||
info!(console, "PM", " [OK] Simple alloc/free");
|
||||
|
||||
let b3 = buddy.alloc(1).expect("buddy: alloc order 1");
|
||||
assert_eq!(b3, 0, "buddy: first order 1 block at 0");
|
||||
let b4 = buddy.alloc(1).expect("buddy: alloc order 1 #2");
|
||||
assert_eq!(b4, 2, "buddy: second order 1 block at 2");
|
||||
buddy.free(b3, 1);
|
||||
buddy.free(b4, 1);
|
||||
let b5 = buddy.alloc(2).expect("buddy: order 2 after coalescing");
|
||||
assert_eq!(b5, 0, "buddy: coalesced block at 0");
|
||||
buddy.free(b5, 2);
|
||||
info!(console, "PM", " [OK] Coalescing across orders (O(1) intrusive list)");
|
||||
|
||||
let mut allocs = Vec::new();
|
||||
loop {
|
||||
match buddy.alloc(0) {
|
||||
Some(idx) => allocs.push(idx),
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
assert!(buddy.is_exhausted(), "buddy: should be exhausted");
|
||||
info!(console, "PM", " [OK] Exhaustion after {} allocs", allocs.len());
|
||||
|
||||
for (i, idx) in allocs.iter().enumerate() {
|
||||
buddy.free(*idx, 0);
|
||||
}
|
||||
assert_eq!(buddy.free_pages(), buddy_test_pages, "buddy: not fully recovered");
|
||||
info!(console, "PM", " [OK] Full recovery after freeing {} pages", allocs.len());
|
||||
|
||||
info!(console, "PM", "~) BUDDY DIRECT TEST PASSED (~");
|
||||
|
||||
info!(console, "PM", "=-= PMM Tree Bitmap Alloc/Free Stress Test =-=");
|
||||
|
||||
const PMM_TEST_COUNT: usize = 64;
|
||||
let mut frames = [PhysAddr(0); PMM_TEST_COUNT];
|
||||
for i in 0..PMM_TEST_COUNT {
|
||||
frames[i] = mem::pmm::alloc_frame().expect("PMM tree: OOM during alloc test");
|
||||
}
|
||||
info!(console, "PM", " [OK] Allocated {} frames", PMM_TEST_COUNT);
|
||||
|
||||
for f in &frames {
|
||||
mem::pmm::free_frame(*f);
|
||||
}
|
||||
info!(console, "PM", " [OK] Freed {} frames back", PMM_TEST_COUNT);
|
||||
|
||||
for i in 0..PMM_TEST_COUNT {
|
||||
frames[i] = mem::pmm::alloc_frame().expect("PMM tree: OOM after free cycle");
|
||||
}
|
||||
info!(console, "PM", " [OK] Re-allocated {} frames after free cycle", PMM_TEST_COUNT);
|
||||
|
||||
for f in &frames {
|
||||
mem::pmm::free_frame(*f);
|
||||
}
|
||||
info!(console, "PM", "~) PMM TREE BITMAP TEST PASSED (~");
|
||||
|
||||
let logo = r#"
|
||||
###########
|
||||
##################
|
||||
@@ -262,6 +476,7 @@ unsafe extern "C" fn kmain() -> ! {
|
||||
}
|
||||
|
||||
fn hcf() -> ! {
|
||||
unsafe { asm!("cli", options(nomem, nostack, preserves_flags)); }
|
||||
loop {
|
||||
unsafe { asm!("hlt", options(nomem, nostack, preserves_flags)); }
|
||||
}
|
||||
@@ -269,7 +484,6 @@ fn hcf() -> ! {
|
||||
|
||||
#[panic_handler]
|
||||
fn rust_panic(info: &core::panic::PanicInfo) -> ! {
|
||||
let mut sp = unsafe { debug::serial::SerialPort::init() };
|
||||
let _ = writeln!(sp, "KERNEL PANIC: {:?}", info);
|
||||
debug::serial::write_global(format_args!("KERNEL PANIC: {:?}\n", info));
|
||||
hcf();
|
||||
}
|
||||
|
||||
@@ -1,3 +1,19 @@
|
||||
use core::sync::atomic::{AtomicU64, Ordering};
|
||||
|
||||
static HHDM_OFFSET: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
/// Store the Higher-Half Direct Map offset obtained from the bootloader.
|
||||
/// Must be called once during early boot, before any address translation.
|
||||
pub fn init_hhdm(offset: u64) {
|
||||
HHDM_OFFSET.store(offset, Ordering::Release);
|
||||
}
|
||||
|
||||
/// Return the HHDM offset (kernel virtual base for physical memory).
|
||||
#[inline]
|
||||
pub fn get_hhdm() -> u64 {
|
||||
HHDM_OFFSET.load(Ordering::Relaxed)
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
|
||||
#[repr(transparent)]
|
||||
pub struct PhysAddr(pub u64);
|
||||
@@ -8,8 +24,8 @@ pub struct VirtAddr(pub u64);
|
||||
|
||||
impl PhysAddr {
|
||||
/// Convert physical address to virtual via HHDM offset
|
||||
pub fn to_virt(self, hhdm_offset: u64) -> VirtAddr {
|
||||
VirtAddr(self.0 + hhdm_offset)
|
||||
pub fn to_virt(self) -> VirtAddr {
|
||||
VirtAddr(self.0 + get_hhdm())
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
@@ -24,9 +40,10 @@ impl PhysAddr {
|
||||
|
||||
impl VirtAddr {
|
||||
#[allow(dead_code)]
|
||||
pub fn to_phys(self, hhdm_offset: u64) -> Option<PhysAddr> {
|
||||
if self.0 < hhdm_offset { return None; }
|
||||
Some(PhysAddr(self.0 - hhdm_offset))
|
||||
pub fn to_phys(self) -> Option<PhysAddr> {
|
||||
let hhdm = get_hhdm();
|
||||
if self.0 < hhdm { return None; }
|
||||
Some(PhysAddr(self.0 - hhdm))
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
|
||||
@@ -49,41 +49,135 @@ impl<A> DerefMut for LockedGuard<'_, A> {
|
||||
fn deref_mut(&mut self) -> &mut Self::Target { self.data }
|
||||
}
|
||||
|
||||
pub struct BumpAllocator {
|
||||
start: usize,
|
||||
end: usize,
|
||||
next: usize,
|
||||
struct ListNode {
|
||||
next: Option<&'static mut ListNode>,
|
||||
}
|
||||
|
||||
impl BumpAllocator {
|
||||
const BLOCK_SIZES: &[usize] = &[8, 16, 32, 64, 128, 256, 512, 1024, 2048];
|
||||
|
||||
struct LargeBlockNode {
|
||||
size: usize,
|
||||
next: Option<&'static mut LargeBlockNode>,
|
||||
}
|
||||
|
||||
pub struct SlabAllocator {
|
||||
list_heads: [Option<&'static mut ListNode>; BLOCK_SIZES.len()],
|
||||
large_block_free: Option<&'static mut LargeBlockNode>,
|
||||
heap_start: usize,
|
||||
heap_end: usize,
|
||||
next_bump: usize,
|
||||
}
|
||||
|
||||
impl SlabAllocator {
|
||||
pub const fn new() -> Self {
|
||||
Self { start: 0, end: 0, next: 0 }
|
||||
Self {
|
||||
list_heads: [None, None, None, None, None, None, None, None, None],
|
||||
large_block_free: None,
|
||||
heap_start: 0,
|
||||
heap_end: 0,
|
||||
next_bump: 0,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn init(&mut self, start: usize, size: usize) {
|
||||
self.start = start;
|
||||
self.next = start;
|
||||
self.end = start + size;
|
||||
self.heap_start = start;
|
||||
self.next_bump = start;
|
||||
self.heap_end = start + size;
|
||||
}
|
||||
|
||||
fn list_index(layout: &Layout) -> Option<usize> {
|
||||
let required_block_size = layout.size().max(layout.align());
|
||||
BLOCK_SIZES.iter().position(|&s| s >= required_block_size)
|
||||
}
|
||||
|
||||
fn fallback_alloc(&mut self, layout: Layout) -> *mut u8 {
|
||||
let size = layout.size().max(layout.align());
|
||||
|
||||
if size > 2048 {
|
||||
let mut field: *mut Option<&'static mut LargeBlockNode> = &mut self.large_block_free;
|
||||
unsafe {
|
||||
while let Some(ref mut node) = *field {
|
||||
if node.size >= size {
|
||||
let node_ptr: *mut LargeBlockNode = *node;
|
||||
let next = node.next.take();
|
||||
*field = next;
|
||||
return node_ptr as *mut u8;
|
||||
}
|
||||
field = &mut node.next;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
unsafe impl GlobalAlloc for Locked<BumpAllocator> {
|
||||
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
|
||||
let mut bump = self.lock();
|
||||
let alloc_start = (self.next_bump + layout.align() - 1) & !(layout.align() - 1);
|
||||
let alloc_end = alloc_start.checked_add(layout.size()).unwrap_or(self.heap_end + 1);
|
||||
|
||||
let alloc_start = (bump.next + layout.align() - 1) & !(layout.align() - 1);
|
||||
let alloc_end = alloc_start + layout.size();
|
||||
|
||||
if alloc_end > bump.end {
|
||||
if alloc_end > self.heap_end {
|
||||
null_mut()
|
||||
} else {
|
||||
bump.next = alloc_end;
|
||||
self.next_bump = alloc_end;
|
||||
alloc_start as *mut u8
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
unsafe fn dealloc(&self, _ptr: *mut u8, _layout: Layout) {}
|
||||
unsafe impl GlobalAlloc for Locked<SlabAllocator> {
|
||||
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
|
||||
let mut allocator = self.lock();
|
||||
|
||||
match SlabAllocator::list_index(&layout) {
|
||||
Some(index) => {
|
||||
match allocator.list_heads[index].take() {
|
||||
Some(node) => {
|
||||
allocator.list_heads[index] = node.next.take();
|
||||
node as *mut ListNode as *mut u8
|
||||
}
|
||||
None => {
|
||||
let block_size = BLOCK_SIZES[index];
|
||||
let block_align = block_size;
|
||||
let layout = Layout::from_size_align(block_size, block_align).unwrap();
|
||||
allocator.fallback_alloc(layout)
|
||||
}
|
||||
}
|
||||
}
|
||||
None => allocator.fallback_alloc(layout)
|
||||
}
|
||||
}
|
||||
|
||||
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
|
||||
let mut allocator = self.lock();
|
||||
|
||||
match SlabAllocator::list_index(&layout) {
|
||||
Some(index) => {
|
||||
let new_node = ListNode {
|
||||
next: allocator.list_heads[index].take(),
|
||||
};
|
||||
|
||||
assert!(layout.size() >= core::mem::size_of::<ListNode>());
|
||||
|
||||
let new_node_ptr = ptr as *mut ListNode;
|
||||
|
||||
// Production-fix для Rust 2024: явная изоляция unsafe-операций
|
||||
unsafe {
|
||||
new_node_ptr.write(new_node);
|
||||
allocator.list_heads[index] = Some(&mut *new_node_ptr);
|
||||
}
|
||||
}
|
||||
None => {
|
||||
// Large block: add to free list for reuse
|
||||
let size = layout.size().max(layout.align());
|
||||
let new_node = LargeBlockNode {
|
||||
size,
|
||||
next: allocator.large_block_free.take(),
|
||||
};
|
||||
let new_node_ptr = ptr as *mut LargeBlockNode;
|
||||
unsafe {
|
||||
new_node_ptr.write(new_node);
|
||||
allocator.large_block_free = Some(&mut *new_node_ptr);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[global_allocator]
|
||||
pub static ALLOCATOR: Locked<BumpAllocator> = Locked::new(BumpAllocator::new());
|
||||
pub static ALLOCATOR: Locked<SlabAllocator> = Locked::new(SlabAllocator::new());
|
||||
|
||||
233
kernel/src/mem/buddy.rs
Normal file
233
kernel/src/mem/buddy.rs
Normal file
@@ -0,0 +1,233 @@
|
||||
use crate::mem::address::{PhysAddr, VirtAddr};
|
||||
|
||||
pub const MAX_ORDER: usize = 11;
|
||||
|
||||
const NEXT_SENTINEL: usize = usize::MAX;
|
||||
|
||||
#[inline]
|
||||
pub fn order_for(page_count: usize) -> usize {
|
||||
if page_count <= 1 {
|
||||
0
|
||||
} else {
|
||||
usize::BITS as usize - (page_count - 1).leading_zeros() as usize
|
||||
}
|
||||
}
|
||||
|
||||
pub struct BuddyAllocator {
|
||||
free_heads: [usize; MAX_ORDER + 1],
|
||||
total_pages: usize,
|
||||
free_pages: usize,
|
||||
base_phys: u64,
|
||||
}
|
||||
|
||||
impl BuddyAllocator {
|
||||
#[inline]
|
||||
fn page_virt(&self, idx: usize) -> VirtAddr {
|
||||
PhysAddr(self.base_phys + idx as u64 * 4096).to_virt()
|
||||
}
|
||||
|
||||
unsafe fn write_node(&self, idx: usize, next: usize, prev: usize) {
|
||||
unsafe {
|
||||
let ptr = self.page_virt(idx).as_mut_ptr::<usize>();
|
||||
ptr.write(next);
|
||||
ptr.add(1).write(prev);
|
||||
}
|
||||
}
|
||||
|
||||
unsafe fn read_next(&self, idx: usize) -> usize {
|
||||
unsafe { self.page_virt(idx).as_ptr::<usize>().read() }
|
||||
}
|
||||
|
||||
unsafe fn read_prev(&self, idx: usize) -> usize {
|
||||
unsafe { self.page_virt(idx).as_ptr::<usize>().add(1).read() }
|
||||
}
|
||||
|
||||
fn flist_push(&mut self, order: usize, idx: usize) {
|
||||
let head = self.free_heads[order];
|
||||
unsafe {
|
||||
self.write_node(idx, head, NEXT_SENTINEL);
|
||||
if head != NEXT_SENTINEL {
|
||||
self.write_node(head, self.read_next(head), idx);
|
||||
}
|
||||
}
|
||||
self.free_heads[order] = idx;
|
||||
}
|
||||
|
||||
fn flist_remove(&mut self, order: usize, idx: usize) {
|
||||
unsafe {
|
||||
let next = self.read_next(idx);
|
||||
let prev = self.read_prev(idx);
|
||||
|
||||
self.write_node(idx, NEXT_SENTINEL, NEXT_SENTINEL);
|
||||
|
||||
if prev != NEXT_SENTINEL {
|
||||
self.write_node(prev, next, self.read_prev(prev));
|
||||
} else {
|
||||
self.free_heads[order] = next;
|
||||
}
|
||||
|
||||
if next != NEXT_SENTINEL {
|
||||
self.write_node(next, self.read_next(next), prev);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn flist_pop(&mut self, order: usize) -> Option<usize> {
|
||||
let head = self.free_heads[order];
|
||||
if head == NEXT_SENTINEL {
|
||||
return None;
|
||||
}
|
||||
self.flist_remove(order, head);
|
||||
Some(head)
|
||||
}
|
||||
|
||||
fn flist_contains(&self, order: usize, idx: usize) -> bool {
|
||||
if self.free_heads[order] == idx {
|
||||
return true;
|
||||
}
|
||||
unsafe { self.read_prev(idx) != NEXT_SENTINEL }
|
||||
}
|
||||
|
||||
pub fn new(total_pages: usize, base_phys: u64) -> Self {
|
||||
let mut this = Self {
|
||||
free_heads: [NEXT_SENTINEL; MAX_ORDER + 1],
|
||||
total_pages,
|
||||
free_pages: 0,
|
||||
base_phys,
|
||||
};
|
||||
|
||||
if total_pages == 0 {
|
||||
return this;
|
||||
}
|
||||
|
||||
let mut idx = 0usize;
|
||||
while idx < total_pages {
|
||||
let remaining = total_pages - idx;
|
||||
|
||||
let align_order = if idx == 0 {
|
||||
MAX_ORDER
|
||||
} else {
|
||||
(idx.trailing_zeros() as usize).min(MAX_ORDER)
|
||||
};
|
||||
|
||||
let size_order = (usize::BITS as usize - 1)
|
||||
- remaining.leading_zeros() as usize;
|
||||
|
||||
let order = MAX_ORDER.min(align_order).min(size_order);
|
||||
let block_size = 1usize << order;
|
||||
|
||||
this.flist_push(order, idx);
|
||||
this.free_pages += block_size;
|
||||
idx += block_size;
|
||||
}
|
||||
|
||||
this
|
||||
}
|
||||
|
||||
pub fn alloc(&mut self, order: usize) -> Option<usize> {
|
||||
if order > MAX_ORDER {
|
||||
return None;
|
||||
}
|
||||
|
||||
let found_order = (order..=MAX_ORDER)
|
||||
.find(|&o| self.free_heads[o] != NEXT_SENTINEL)?;
|
||||
|
||||
let block_idx = self.flist_pop(found_order)
|
||||
.expect("buddy: free_list non-empty but flist_pop returned None");
|
||||
|
||||
self.free_pages -= 1 << found_order;
|
||||
|
||||
let mut cur_order = found_order;
|
||||
while cur_order > order {
|
||||
cur_order -= 1;
|
||||
let buddy_idx = block_idx + (1 << cur_order);
|
||||
self.flist_push(cur_order, buddy_idx);
|
||||
self.free_pages += 1 << cur_order;
|
||||
}
|
||||
|
||||
Some(block_idx)
|
||||
}
|
||||
|
||||
pub fn alloc_pages(&mut self, page_count: usize) -> Option<(usize, usize)> {
|
||||
if page_count == 0 {
|
||||
return None;
|
||||
}
|
||||
let order = order_for(page_count);
|
||||
if order > MAX_ORDER {
|
||||
return None;
|
||||
}
|
||||
self.alloc(order).map(|idx| (idx, order))
|
||||
}
|
||||
|
||||
pub fn free(&mut self, mut block_idx: usize, mut order: usize) {
|
||||
debug_assert!(
|
||||
order <= MAX_ORDER,
|
||||
"buddy::free: order {} > MAX_ORDER {}",
|
||||
order,
|
||||
MAX_ORDER
|
||||
);
|
||||
debug_assert!(
|
||||
block_idx.checked_add(1 << order).map_or(false, |e| e <= self.total_pages),
|
||||
"buddy::free: block [{}, +{}] out of range (total_pages={})",
|
||||
block_idx,
|
||||
1usize << order,
|
||||
self.total_pages
|
||||
);
|
||||
|
||||
while order < MAX_ORDER {
|
||||
let buddy_idx = block_idx ^ (1 << order);
|
||||
|
||||
let buddy_end = match buddy_idx.checked_add(1 << order) {
|
||||
Some(e) => e,
|
||||
None => break,
|
||||
};
|
||||
if buddy_end > self.total_pages {
|
||||
break;
|
||||
}
|
||||
|
||||
if self.flist_contains(order, buddy_idx) {
|
||||
self.flist_remove(order, buddy_idx);
|
||||
self.free_pages -= 1 << order;
|
||||
block_idx = block_idx.min(buddy_idx);
|
||||
order += 1;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
self.free_pages += 1 << order;
|
||||
self.flist_push(order, block_idx);
|
||||
}
|
||||
|
||||
#[inline]
|
||||
pub fn free_pages(&self) -> usize {
|
||||
self.free_pages
|
||||
}
|
||||
|
||||
#[inline]
|
||||
pub fn total_pages(&self) -> usize {
|
||||
self.total_pages
|
||||
}
|
||||
|
||||
#[inline]
|
||||
pub fn is_exhausted(&self) -> bool {
|
||||
self.free_pages == 0
|
||||
}
|
||||
|
||||
pub fn stats(&self) -> [(usize, usize); MAX_ORDER + 1] {
|
||||
core::array::from_fn(|o| {
|
||||
let count = if self.free_heads[o] == NEXT_SENTINEL {
|
||||
0
|
||||
} else {
|
||||
let mut cnt = 0;
|
||||
let mut cur = self.free_heads[o];
|
||||
while cur != NEXT_SENTINEL {
|
||||
cnt += 1;
|
||||
cur = unsafe { self.read_next(cur) };
|
||||
}
|
||||
cnt
|
||||
};
|
||||
(o, count)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,16 +1,24 @@
|
||||
pub mod pmm;
|
||||
pub mod address;
|
||||
pub mod paging;
|
||||
pub mod allocator;
|
||||
pub mod buddy;
|
||||
pub mod paging;
|
||||
pub mod pm_manages;
|
||||
pub mod pmm;
|
||||
pub mod vmm;
|
||||
pub mod pm_router;
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub fn init(memmap: &limine::response::MemoryMapResponse, hhdm_offset: u64) {
|
||||
unsafe {
|
||||
pmm::BitmapPMM::init(memmap, hhdm_offset);
|
||||
}
|
||||
pub fn init_pmm(memmap: &limine::response::MemoryMapResponse) {
|
||||
unsafe { pmm::BitmapPMM::init(memmap); }
|
||||
}
|
||||
|
||||
pub fn init_cpu_features() {
|
||||
vmm::init_cpu_features();
|
||||
}
|
||||
|
||||
pub fn init_vmm(pml4_phys: address::PhysAddr) {
|
||||
vmm::init_kernel_space(pml4_phys);
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub fn get_stats() -> (usize, usize) {
|
||||
pmm::get_stats()
|
||||
}
|
||||
|
||||
@@ -16,69 +16,242 @@ bitflags! {
|
||||
const DIRTY = 1 << 6;
|
||||
const HUGE_PAGE = 1 << 7;
|
||||
const GLOBAL = 1 << 8;
|
||||
const COW = 1 << 9;
|
||||
const NO_EXECUTE = 1 << 63;
|
||||
}
|
||||
}
|
||||
|
||||
/// Mask for the physical frame address stored in a PTE (bits 51:12).
|
||||
const PTE_ADDR_MASK: u64 = 0x000F_FFFF_FFFF_F000;
|
||||
|
||||
#[repr(C, align(4096))]
|
||||
pub struct PageTable {
|
||||
entries: [u64; 512],
|
||||
}
|
||||
|
||||
impl PageTable {
|
||||
pub fn map_region(&mut self, virt: VirtAddr, phys: PhysAddr, size: u64, flags: PageTableFlags, hhdm: u64) {
|
||||
//Bulk mapping
|
||||
|
||||
/// Map a contiguous physical range to a contiguous virtual range.
|
||||
/// Allocates intermediate page-table pages from the PMM as needed.
|
||||
pub fn map_region(
|
||||
&mut self,
|
||||
virt: VirtAddr,
|
||||
phys: PhysAddr,
|
||||
size: u64,
|
||||
flags: PageTableFlags,
|
||||
) {
|
||||
let pages = size.div_ceil(4096);
|
||||
for i in 0..pages {
|
||||
let offset = i * 4096;
|
||||
self.map_page(VirtAddr(virt.0 + offset), PhysAddr(phys.0 + offset), flags, hhdm);
|
||||
self.map_page(VirtAddr(virt.0 + offset), PhysAddr(phys.0 + offset), flags);
|
||||
}
|
||||
}
|
||||
|
||||
/// Loads the page table into the CR3 register
|
||||
pub unsafe fn activate(&self, phys_addr: PhysAddr) {
|
||||
pub fn get_flags(&self, virt: VirtAddr) -> Option<PageTableFlags> {
|
||||
let p4_idx = ((virt.0 >> 39) & 0x1FF) as usize;
|
||||
let p3_idx = ((virt.0 >> 30) & 0x1FF) as usize;
|
||||
let p2_idx = ((virt.0 >> 21) & 0x1FF) as usize;
|
||||
let p1_idx = ((virt.0 >> 12) & 0x1FF) as usize;
|
||||
|
||||
macro_rules! descend_ref {
|
||||
($entry:expr) => {{
|
||||
let e = $entry;
|
||||
if e & PageTableFlags::PRESENT.bits() == 0 { return None; }
|
||||
unsafe { &*PhysAddr(e & PTE_ADDR_MASK).to_virt().as_mut_ptr::<PageTable>() }
|
||||
}};
|
||||
}
|
||||
|
||||
let p3 = descend_ref!(self.entries[p4_idx]);
|
||||
let p3e = p3.entries[p3_idx];
|
||||
if p3e & PageTableFlags::HUGE_PAGE.bits() != 0 {
|
||||
return Some(PageTableFlags::from_bits_truncate(p3e));
|
||||
}
|
||||
|
||||
let p2 = descend_ref!(p3e);
|
||||
let p2e = p2.entries[p2_idx];
|
||||
if p2e & PageTableFlags::HUGE_PAGE.bits() != 0 {
|
||||
return Some(PageTableFlags::from_bits_truncate(p2e));
|
||||
}
|
||||
|
||||
let p1 = descend_ref!(p2e);
|
||||
let p1e = p1.entries[p1_idx];
|
||||
if p1e & PageTableFlags::PRESENT.bits() == 0 { return None; }
|
||||
|
||||
Some(PageTableFlags::from_bits_truncate(p1e))
|
||||
}
|
||||
|
||||
pub fn update_flags(&mut self, virt: VirtAddr, flags: PageTableFlags) -> Result<(), ()> {
|
||||
let Some(p1) = self.walk_to_p1_mut(virt, false) else { return Err(()); };
|
||||
let p1_idx = ((virt.0 >> 12) & 0x1FF) as usize;
|
||||
let entry = p1.entries[p1_idx];
|
||||
|
||||
if entry & PageTableFlags::PRESENT.bits() == 0 { return Err(()); }
|
||||
|
||||
p1.entries[p1_idx] = (entry & PTE_ADDR_MASK) | flags.bits();
|
||||
|
||||
unsafe {
|
||||
asm!(
|
||||
"mov cr3, {0}",
|
||||
"jmp 2f",
|
||||
"2:",
|
||||
in(reg) phys_addr.0,
|
||||
options(nostack, preserves_flags)
|
||||
);
|
||||
asm!("invlpg [{}]", in(reg) virt.0, options(nostack, preserves_flags));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn map_page(&mut self, virt: VirtAddr, phys: PhysAddr, flags: PageTableFlags, hhdm_offset: u64) {
|
||||
let p4_idx = (virt.0 >> 39) & 0x1FF;
|
||||
let p3_idx = (virt.0 >> 30) & 0x1FF;
|
||||
let p2_idx = (virt.0 >> 21) & 0x1FF;
|
||||
let p1_idx = (virt.0 >> 12) & 0x1FF;
|
||||
|
||||
let p3 = self.get_or_create_next_table(p4_idx as usize, hhdm_offset);
|
||||
let p2 = p3.get_or_create_next_table(p3_idx as usize, hhdm_offset);
|
||||
let p1 = p2.get_or_create_next_table(p2_idx as usize, hhdm_offset);
|
||||
//Single-page operations
|
||||
|
||||
p1.entries[p1_idx as usize] = phys.0 | flags.bits();
|
||||
/// Map a single 4 KiB page.
|
||||
/// Allocates intermediate PT pages from the PMM if they do not exist.
|
||||
pub fn map_page(
|
||||
&mut self,
|
||||
virt: VirtAddr,
|
||||
phys: PhysAddr,
|
||||
flags: PageTableFlags,
|
||||
) {
|
||||
let p1 = self
|
||||
.walk_to_p1_mut(virt, true /* create */)
|
||||
.expect("map_page: OOM allocating intermediate page-table pages");
|
||||
|
||||
let p1_idx = ((virt.0 >> 12) & 0x1FF) as usize;
|
||||
p1.entries[p1_idx] = phys.0 | flags.bits();
|
||||
|
||||
unsafe {
|
||||
asm!("invlpg [{}]", in(reg) virt.0, options(nostack, preserves_flags));
|
||||
}
|
||||
}
|
||||
|
||||
fn get_or_create_next_table(&mut self, index: usize, hhdm: u64) -> &mut Self {
|
||||
if self.entries[index] & PageTableFlags::PRESENT.bits() == 0 {
|
||||
let pt_phys = pmm_alloc().expect("VMM: Out of memory for page tables");
|
||||
let pt_virt = pt_phys.to_virt(hhdm);
|
||||
/// Clear the PTE for `virt` and issue an `INVLPG`.
|
||||
///
|
||||
/// Does **not** free the underlying physical frame — that is the caller's
|
||||
/// responsibility (e.g. `VmaBacking::do_unmap`).
|
||||
/// Silently returns if any level is not present (idempotent).
|
||||
pub fn unmap_page(&mut self, virt: VirtAddr) {
|
||||
let Some(p1) = self.walk_to_p1_mut(virt, false /* no create */) else {
|
||||
return; // already absent — nothing to do
|
||||
};
|
||||
let p1_idx = ((virt.0 >> 12) & 0x1FF) as usize;
|
||||
if p1.entries[p1_idx] & PageTableFlags::PRESENT.bits() == 0 {
|
||||
return;
|
||||
}
|
||||
p1.entries[p1_idx] = 0;
|
||||
unsafe {
|
||||
asm!("invlpg [{}]", in(reg) virt.0, options(nostack, preserves_flags));
|
||||
}
|
||||
}
|
||||
|
||||
/// Walk four page-table levels to translate `virt` → physical address.
|
||||
///
|
||||
/// Handles 1 GiB and 2 MiB huge pages transparently.
|
||||
/// Returns `None` if any level is absent or the page is not present.
|
||||
pub fn translate(&self, virt: VirtAddr) -> Option<PhysAddr> {
|
||||
let p4_idx = ((virt.0 >> 39) & 0x1FF) as usize;
|
||||
let p3_idx = ((virt.0 >> 30) & 0x1FF) as usize;
|
||||
let p2_idx = ((virt.0 >> 21) & 0x1FF) as usize;
|
||||
let p1_idx = ((virt.0 >> 12) & 0x1FF) as usize;
|
||||
|
||||
// Helper: safely follow a PTE to the next level.
|
||||
macro_rules! descend_ref {
|
||||
($entry:expr) => {{
|
||||
let e = $entry;
|
||||
if e & PageTableFlags::PRESENT.bits() == 0 { return None; }
|
||||
unsafe { &*PhysAddr(e & PTE_ADDR_MASK).to_virt().as_mut_ptr::<PageTable>() }
|
||||
}};
|
||||
}
|
||||
|
||||
let p3 = descend_ref!(self.entries[p4_idx]);
|
||||
|
||||
// 1 GiB page (PD pointer with HUGE_PAGE set)
|
||||
let p3e = p3.entries[p3_idx];
|
||||
if p3e & PageTableFlags::HUGE_PAGE.bits() != 0 {
|
||||
let base = p3e & 0x000F_FFFF_C000_0000; // bits 51:30
|
||||
return Some(PhysAddr(base | (virt.0 & 0x3FFF_FFFF)));
|
||||
}
|
||||
|
||||
let p2 = descend_ref!(p3e);
|
||||
|
||||
// 2 MiB page (PD entry with HUGE_PAGE set)
|
||||
let p2e = p2.entries[p2_idx];
|
||||
if p2e & PageTableFlags::HUGE_PAGE.bits() != 0 {
|
||||
let base = p2e & 0x000F_FFFF_FFE0_0000; // bits 51:21
|
||||
return Some(PhysAddr(base | (virt.0 & 0x001F_FFFF)));
|
||||
}
|
||||
|
||||
let p1 = descend_ref!(p2e);
|
||||
let p1e = p1.entries[p1_idx];
|
||||
if p1e & PageTableFlags::PRESENT.bits() == 0 { return None; }
|
||||
|
||||
Some(PhysAddr((p1e & PTE_ADDR_MASK) | (virt.0 & 0xFFF)))
|
||||
}
|
||||
|
||||
// CR3
|
||||
|
||||
/// Load this page table into CR3 (full TLB flush, no PCID).
|
||||
///
|
||||
/// Prefer `AddressSpace::activate` (which uses PCID/INVPCID) for process
|
||||
/// context switches. Use this only for early boot before PCID is enabled.
|
||||
pub unsafe fn activate(&self, phys_addr: PhysAddr) {
|
||||
unsafe {
|
||||
asm!(
|
||||
"mov cr3, {0}",
|
||||
in(reg) phys_addr.0,
|
||||
options(nostack, preserves_flags),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
//Private walk helpers
|
||||
|
||||
impl PageTable {
|
||||
/// Walk (or create) the path P4 → P3 → P2 → P1, returning a mutable
|
||||
/// reference to the P1 (page table, final level).
|
||||
///
|
||||
/// If `create` is `false` and any intermediate entry is absent, returns `None`.
|
||||
/// If `create` is `true`, allocates missing intermediate pages from the PMM.
|
||||
fn walk_to_p1_mut(&mut self, virt: VirtAddr, create: bool)
|
||||
-> Option<&mut Self>
|
||||
{
|
||||
let p4_idx = ((virt.0 >> 39) & 0x1FF) as usize;
|
||||
let p3_idx = ((virt.0 >> 30) & 0x1FF) as usize;
|
||||
let p2_idx = ((virt.0 >> 21) & 0x1FF) as usize;
|
||||
|
||||
let p3 = self.get_or_create_next_table(p4_idx, create)?;
|
||||
let p2 = p3.get_or_create_next_table(p3_idx, create)?;
|
||||
p2.get_or_create_next_table(p2_idx, create)
|
||||
}
|
||||
|
||||
/// Return a mutable reference to the next-level table at `index`.
|
||||
///
|
||||
/// - If the entry is present, follows the pointer.
|
||||
/// - If absent and `create`, allocates a zeroed page and installs it.
|
||||
/// - If absent and `!create`, returns `None`.
|
||||
fn get_or_create_next_table(
|
||||
&mut self,
|
||||
index: usize,
|
||||
create: bool,
|
||||
) -> Option<&mut Self> {
|
||||
let entry = self.entries[index];
|
||||
if entry & PageTableFlags::PRESENT.bits() == 0 {
|
||||
if !create { return None; }
|
||||
|
||||
let pt_phys = pmm_alloc().expect("VMM: OOM allocating page-table page");
|
||||
let pt_virt = pt_phys.to_virt();
|
||||
unsafe { core::ptr::write_bytes(pt_virt.as_mut_ptr::<u8>(), 0, 4096); }
|
||||
|
||||
self.entries[index] = pt_phys.0 | (PageTableFlags::PRESENT | PageTableFlags::WRITABLE | PageTableFlags::USER).bits();
|
||||
// Install with USER so both kernel and user pages can live under it;
|
||||
// actual user/kernel distinction is enforced at the P1 level.
|
||||
self.entries[index] = pt_phys.0
|
||||
| (PageTableFlags::PRESENT | PageTableFlags::WRITABLE | PageTableFlags::USER).bits();
|
||||
}
|
||||
let next_pt_phys = PhysAddr(self.entries[index] & 0x000F_FFFF_FFFF_F000);
|
||||
unsafe { &mut *next_pt_phys.to_virt(hhdm).as_mut_ptr() }
|
||||
|
||||
let next_phys = PhysAddr(self.entries[index] & PTE_ADDR_MASK);
|
||||
Some(unsafe { &mut *next_phys.to_virt().as_mut_ptr::<Self>() })
|
||||
}
|
||||
}
|
||||
|
||||
//PMM shim
|
||||
|
||||
/// Allocate a single physical frame for page-table use.
|
||||
/// This thin wrapper avoids a direct dependency cycle between paging ↔ pmm.
|
||||
pub fn pmm_alloc() -> Option<PhysAddr> {
|
||||
PMM.lock().as_mut()?.alloc_frame()
|
||||
}
|
||||
|
||||
|
||||
@@ -1,25 +1,457 @@
|
||||
use crate::cap::{Capability, CapObject, Relation, CapRights};
|
||||
use crate::mem::address::PhysAddr;
|
||||
//!
|
||||
//! Distributed physical-memory actor that owns a fixed physical capital range
|
||||
//! and processes allocation requests asynchronously via a lock-free MPSC queue.
|
||||
//!
|
||||
//! ## Key changes vs. previous version
|
||||
//!
|
||||
//! 1. **Real allocator**: `PMActor` now contains a `BuddyAllocator` instead of a
|
||||
//! bare page counter. Allocations produce real relative frame indices that
|
||||
//! translate to absolute `PhysAddr` values.
|
||||
//!
|
||||
//! 2. **Closed response loop**: `process_messages()` returns `Vec<PMResponse>`.
|
||||
//! The caller (scheduler / boot code) routes each `PMResponse` to the
|
||||
//! requesting entity identified by `channel_id`. `channel_id = 0` means
|
||||
//! "discard" and is safe to ignore.
|
||||
//!
|
||||
//! 3. **Revised message packing** — fits all fields into a single `u64` without
|
||||
//! truncating critical identifiers:
|
||||
//! ```text
|
||||
//! bits 63:56 OPCODE (8 bits) 1=Alloc, 2=Free, 3=Carve
|
||||
//! bits 55:40 CHANNEL_ID (16 bits) response routing; 0 = discard
|
||||
//! bits 39:20 ARG1 (20 bits) size_pages / local_frame_idx / offset_pages
|
||||
//! bits 19: 0 ARG2 (20 bits) token_sig (low 20 bits) / size_pages
|
||||
//! ```
|
||||
//! Maximum ARG size: 2^20 = 1 048 576 pages = 4 GiB per allocation.
|
||||
//! Maximum CHANNEL_ID: 65 535 simultaneous pending requesters.
|
||||
//!
|
||||
//! ## Ownership model
|
||||
//!
|
||||
//! `PMActor` holds a `Strong` capability over its entire managed range.
|
||||
//! Sub-capabilities carved out via `Allocate` or `Carve` are `Strong` children;
|
||||
//! freeing them (via `Free`) coalesces the buddy tree. The actor **never**
|
||||
//! touches the global `BitmapPMM` — that belongs to the kernel boot path.
|
||||
//!
|
||||
//! ## Ballooning hook
|
||||
//!
|
||||
//! When `BuddyAllocator::alloc_pages` returns `None`, the actor emits a
|
||||
//! `PMResponse::OutOfMemory { channel_id }`. The future ballooning subsystem
|
||||
//! will intercept this before it reaches the requester and negotiate memory
|
||||
//! transfer from a neighbouring actor.
|
||||
|
||||
use alloc::vec::Vec;
|
||||
use core::sync::atomic::{AtomicU64, AtomicUsize, Ordering};
|
||||
|
||||
use crate::cap::{CapObject, CapRights, Capability, Relation};
|
||||
use crate::mem::address::PhysAddr;
|
||||
use crate::mem::buddy::BuddyAllocator;
|
||||
use crate::mem::pmm::PAGE_SIZE;
|
||||
|
||||
extern crate alloc;
|
||||
|
||||
// Message packing constants
|
||||
|
||||
const QUEUE_SIZE: usize = 1024; // power-of-two
|
||||
const QUEUE_MASK: usize = QUEUE_SIZE - 1;
|
||||
|
||||
/// Bit layout of a packed PMRequest u64.
|
||||
mod packing {
|
||||
pub const OPCODE_SHIFT: u32 = 56;
|
||||
pub const CHAN_SHIFT: u32 = 40;
|
||||
pub const ARG1_SHIFT: u32 = 20;
|
||||
// ARG2 sits in bits 19:0 (no shift).
|
||||
|
||||
pub const CHAN_MASK: u64 = 0x0000_FFFF; // 16 bits
|
||||
pub const ARG_MASK: u64 = 0x000F_FFFF; // 20 bits
|
||||
}
|
||||
|
||||
// Request type
|
||||
/// Asynchronous request submitted to a `PMActor` inbox.
|
||||
///
|
||||
/// `channel_id` identifies where the response should be routed.
|
||||
/// Use `channel_id = 0` if you do not need a reply.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum PMRequest {
|
||||
/// Allocate `size_pages` pages; tag the resulting capability with `token_sig`.
|
||||
Allocate {
|
||||
size_pages: usize,
|
||||
token_sig: u32,
|
||||
channel_id: u16,
|
||||
},
|
||||
/// Free a previously allocated block.
|
||||
///
|
||||
/// `local_frame_idx` is the **relative** page index returned inside the
|
||||
/// `Allocated` capability (i.e. `cap.object.phys - managed_range.start`).
|
||||
/// `order` is the buddy order used at allocation time (stored in the cap).
|
||||
Free {
|
||||
local_frame_idx: usize,
|
||||
order: usize,
|
||||
},
|
||||
/// Carve a fixed sub-region without consulting the buddy allocator.
|
||||
/// Used for static layout decisions made at boot (e.g. framebuffer alias).
|
||||
Carve {
|
||||
offset_pages: usize,
|
||||
size_pages: usize,
|
||||
channel_id: u16,
|
||||
},
|
||||
None,
|
||||
}
|
||||
|
||||
impl PMRequest {
|
||||
/// Encode the request into one `u64` atom for the lock-free buffer.
|
||||
///
|
||||
/// Opcode 0 (None) is reserved as the "slot empty" sentinel; `None` returns
|
||||
/// 0 intentionally and the sender rejects it before pushing.
|
||||
#[inline]
|
||||
pub fn pack(self) -> u64 {
|
||||
use packing::*;
|
||||
match self {
|
||||
Self::Allocate { size_pages, token_sig, channel_id } => {
|
||||
(1u64 << OPCODE_SHIFT)
|
||||
| ((channel_id as u64 & CHAN_MASK) << CHAN_SHIFT)
|
||||
| ((size_pages as u64 & ARG_MASK) << ARG1_SHIFT)
|
||||
| (token_sig as u64 & ARG_MASK)
|
||||
}
|
||||
Self::Free { local_frame_idx, order } => {
|
||||
(2u64 << OPCODE_SHIFT)
|
||||
// channel_id not needed — Free has no reply
|
||||
| ((local_frame_idx as u64 & ARG_MASK) << ARG1_SHIFT)
|
||||
| (order as u64 & ARG_MASK)
|
||||
}
|
||||
Self::Carve { offset_pages, size_pages, channel_id } => {
|
||||
(3u64 << OPCODE_SHIFT)
|
||||
| ((channel_id as u64 & CHAN_MASK) << CHAN_SHIFT)
|
||||
| ((offset_pages as u64 & ARG_MASK) << ARG1_SHIFT)
|
||||
| (size_pages as u64 & ARG_MASK)
|
||||
}
|
||||
Self::None => 0,
|
||||
}
|
||||
}
|
||||
|
||||
/// Decode a packed `u64` back into a `PMRequest`.
|
||||
#[inline]
|
||||
pub fn unpack(val: u64) -> Self {
|
||||
use packing::*;
|
||||
let opcode = val >> OPCODE_SHIFT;
|
||||
let channel_id = ((val >> CHAN_SHIFT) & CHAN_MASK) as u16;
|
||||
let arg1 = ((val >> ARG1_SHIFT) & ARG_MASK) as usize;
|
||||
let arg2 = (val & ARG_MASK) as usize;
|
||||
|
||||
match opcode {
|
||||
1 => Self::Allocate {
|
||||
size_pages: arg1,
|
||||
token_sig: arg2 as u32,
|
||||
channel_id,
|
||||
},
|
||||
2 => Self::Free {
|
||||
local_frame_idx: arg1,
|
||||
order: arg2,
|
||||
},
|
||||
3 => Self::Carve {
|
||||
offset_pages: arg1,
|
||||
size_pages: arg2,
|
||||
channel_id,
|
||||
},
|
||||
_ => Self::None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Response type
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct PMResponse {
|
||||
pub channel_id: u16,
|
||||
pub result: PMResult,
|
||||
}
|
||||
|
||||
/// Outcome of a single PM operation.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub enum PMResult {
|
||||
Allocated { cap: Capability, order: usize },
|
||||
OutOfMemory { size_pages: usize },
|
||||
Carved { cap: Capability },
|
||||
Freed { pages_returned: usize },
|
||||
}
|
||||
|
||||
// Lock-free MPSC inbox
|
||||
/// MPSC ring buffer for PMRequest values.
|
||||
///
|
||||
/// Producers (any core, any context) call `send`; the owning PMActor calls
|
||||
/// `pop` — exactly one consumer, no locking on the read side.
|
||||
pub struct PMActorQueue {
|
||||
buffer: [AtomicU64; QUEUE_SIZE],
|
||||
_pad0: [u8; 64],
|
||||
head: AtomicUsize,
|
||||
_pad1: [u8; 64],
|
||||
tail: AtomicUsize,
|
||||
}
|
||||
|
||||
impl PMActorQueue {
|
||||
pub const fn new() -> Self {
|
||||
Self {
|
||||
#[allow(clippy::declare_interior_mutable_const)]
|
||||
buffer: [const { AtomicU64::new(0) }; QUEUE_SIZE],
|
||||
_pad0: [0u8; 64],
|
||||
head: AtomicUsize::new(0),
|
||||
_pad1: [0u8; 64],
|
||||
tail: AtomicUsize::new(0),
|
||||
}
|
||||
}
|
||||
|
||||
/// Push a request into the inbox (Multi-Producer path).
|
||||
///
|
||||
/// Returns `Err` only if the ring buffer is full — which should trigger
|
||||
/// backpressure or a ballooning request in production.
|
||||
pub fn send(&self, req: PMRequest) -> Result<(), &'static str> {
|
||||
let packed = req.pack();
|
||||
if packed == 0 {
|
||||
return Err("PMActorQueue::send: PMRequest::None is not sendable");
|
||||
}
|
||||
|
||||
let mut tail = self.tail.load(Ordering::Relaxed);
|
||||
loop {
|
||||
let head = self.head.load(Ordering::Acquire);
|
||||
if tail.wrapping_sub(head) >= QUEUE_SIZE {
|
||||
return Err("PMActorQueue overflow — apply backpressure or balloon");
|
||||
}
|
||||
|
||||
match self.tail.compare_exchange_weak(
|
||||
tail,
|
||||
tail.wrapping_add(1),
|
||||
Ordering::AcqRel,
|
||||
Ordering::Relaxed,
|
||||
) {
|
||||
Ok(_) => {
|
||||
self.buffer[tail & QUEUE_MASK].store(packed, Ordering::Release);
|
||||
return Ok(());
|
||||
}
|
||||
Err(actual) => tail = actual,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Pop one request (Single-Consumer path — actor's own context only).
|
||||
pub fn pop(&self) -> Option<PMRequest> {
|
||||
let head = self.head.load(Ordering::Relaxed);
|
||||
let tail = self.tail.load(Ordering::Acquire);
|
||||
if head == tail {
|
||||
return None;
|
||||
}
|
||||
let packed = self.buffer[head & QUEUE_MASK].load(Ordering::Acquire);
|
||||
self.head.store(head.wrapping_add(1), Ordering::Release);
|
||||
Some(PMRequest::unpack(packed))
|
||||
}
|
||||
}
|
||||
|
||||
// PM Actor
|
||||
/// An autonomous physical-memory actor.
|
||||
///
|
||||
/// Owns a `BuddyAllocator` over its capital range and a lock-free MPSC inbox.
|
||||
/// The actor must be driven externally: when the scheduler grants it CPU time,
|
||||
/// it calls `process_messages()` to drain the inbox and produce `PMResponse`s.
|
||||
///
|
||||
/// # Single-consumer guarantee
|
||||
/// Only `process_messages()` ever mutates `buddy` and `queue.head`.
|
||||
/// This is enforced by taking `&mut self` on `process_messages`.
|
||||
pub struct PMActor {
|
||||
pub actor_id: u64,
|
||||
pub root_untyped: Capability,
|
||||
pub managed_range: (PhysAddr, PhysAddr),
|
||||
queue: PMActorQueue,
|
||||
buddy: BuddyAllocator,
|
||||
}
|
||||
|
||||
impl PMActor {
|
||||
pub fn carve_region(&self, offset: usize, size: usize) -> Capability {
|
||||
if let CapObject::Memory { phys, .. } = self.root_untyped.object {
|
||||
Capability {
|
||||
/// Construct a new actor managing `size_bytes` bytes starting at `start`.
|
||||
///
|
||||
/// `size_bytes` must be a multiple of `PAGE_SIZE`; non-multiples are
|
||||
/// rounded down silently.
|
||||
pub fn new(
|
||||
actor_id: u64,
|
||||
root_cap: Capability,
|
||||
start: PhysAddr,
|
||||
size_bytes: u64,
|
||||
) -> Self {
|
||||
let total_pages = (size_bytes / PAGE_SIZE) as usize;
|
||||
Self {
|
||||
actor_id,
|
||||
root_untyped: root_cap,
|
||||
managed_range: (start, PhysAddr(start.0 + total_pages as u64 * PAGE_SIZE)),
|
||||
queue: PMActorQueue::new(),
|
||||
buddy: BuddyAllocator::new(total_pages, start.0),
|
||||
}
|
||||
}
|
||||
|
||||
//Producer API (callable from any context)
|
||||
|
||||
/// Submit a request to this actor's inbox.
|
||||
///
|
||||
/// Thread-safe (MPSC producer side). Returns `Err` if the inbox is full.
|
||||
#[inline]
|
||||
pub fn submit_request(&self, req: PMRequest) -> Result<(), &'static str> {
|
||||
self.queue.send(req)
|
||||
}
|
||||
|
||||
//Consumer API (actor's own scheduled context)
|
||||
|
||||
/// Drain the inbox and execute all pending requests.
|
||||
///
|
||||
/// Returns a `Vec` of responses that the caller (scheduler) must route to
|
||||
/// the appropriate process or actor mailbox based on `channel_id`.
|
||||
///
|
||||
/// Responses with `channel_id == 0` can be discarded.
|
||||
///
|
||||
/// # No locks held
|
||||
/// `buddy` and `queue.head` are mutated — no other thread touches them.
|
||||
/// The only shared state is the queue's `tail`, which is written by producers
|
||||
/// via `AtomicUsize::compare_exchange_weak`, never by this path.
|
||||
/// Maximum requests to process in a single `process_messages` call.
|
||||
/// Prevents kernel starvation when the inbox is deep.
|
||||
const MAX_MESSAGES_PER_CALL: usize = 64;
|
||||
|
||||
pub fn process_messages(&mut self) -> Vec<PMResponse> {
|
||||
let mut responses = Vec::new();
|
||||
let mut remaining = Self::MAX_MESSAGES_PER_CALL;
|
||||
|
||||
while let Some(req) = self.queue.pop() {
|
||||
remaining -= 1;
|
||||
let resp = match req {
|
||||
PMRequest::Allocate { size_pages, token_sig, channel_id } => {
|
||||
self.handle_allocate(size_pages, token_sig, channel_id)
|
||||
}
|
||||
PMRequest::Free { local_frame_idx, order } => {
|
||||
self.handle_free(local_frame_idx, order)
|
||||
}
|
||||
PMRequest::Carve { offset_pages, size_pages, channel_id } => {
|
||||
self.handle_carve(offset_pages, size_pages, channel_id)
|
||||
}
|
||||
PMRequest::None => continue,
|
||||
};
|
||||
|
||||
if let Some(r) = resp {
|
||||
responses.push(r);
|
||||
}
|
||||
|
||||
if remaining == 0 {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
responses
|
||||
}
|
||||
|
||||
//Introspection
|
||||
|
||||
/// Free pages remaining in this actor's buddy pool.
|
||||
#[inline]
|
||||
pub fn free_pages(&self) -> usize {
|
||||
self.buddy.free_pages()
|
||||
}
|
||||
|
||||
/// Total pages this actor was initialised with.
|
||||
#[inline]
|
||||
pub fn total_pages(&self) -> usize {
|
||||
self.buddy.total_pages()
|
||||
}
|
||||
}
|
||||
|
||||
//Private handler implementations
|
||||
|
||||
impl PMActor {
|
||||
fn handle_allocate(
|
||||
&mut self,
|
||||
size_pages: usize,
|
||||
token_sig: u32,
|
||||
channel_id: u16,
|
||||
) -> Option<PMResponse> {
|
||||
match self.buddy.alloc_pages(size_pages) {
|
||||
Some((rel_idx, order)) => {
|
||||
// Translate relative index → absolute physical address.
|
||||
let phys = PhysAddr(self.managed_range.0.0 + rel_idx as u64 * PAGE_SIZE);
|
||||
|
||||
// Build a strong child capability.
|
||||
// token_sig is widened and XOR'd with actor_id for uniqueness.
|
||||
let cap = Capability {
|
||||
object: CapObject::Memory {
|
||||
phys: PhysAddr(phys.0 + offset as u64),
|
||||
size_pages: size / 4096,
|
||||
phys,
|
||||
// Report actual (rounded-up) page count so the caller
|
||||
// passes the correct `order` back to Free.
|
||||
size_pages: 1 << order,
|
||||
},
|
||||
rights: CapRights::READ | CapRights::WRITE | CapRights::GRANT,
|
||||
relation: Relation::Strong,
|
||||
token_sig: 0xDEAD_BEEF,
|
||||
token_sig: (token_sig as u64) ^ self.actor_id ^ (rel_idx as u64),
|
||||
};
|
||||
|
||||
Some(PMResponse {
|
||||
channel_id,
|
||||
result: PMResult::Allocated { cap, order },
|
||||
})
|
||||
}
|
||||
} else {
|
||||
panic!("PM: Root is not memory!");
|
||||
None => {
|
||||
// TODO: emit a ballooning request to a neighbour actor before
|
||||
// propagating OutOfMemory upward.
|
||||
Some(PMResponse {
|
||||
channel_id,
|
||||
result: PMResult::OutOfMemory { size_pages },
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_free(
|
||||
&mut self,
|
||||
local_frame_idx: usize,
|
||||
order: usize,
|
||||
) -> Option<PMResponse> {
|
||||
// Validate bounds before touching the buddy tree.
|
||||
let block_end = local_frame_idx.saturating_add(1usize << order);
|
||||
if block_end > self.buddy.total_pages() {
|
||||
// Corrupted or spoofed index — ignore silently in release, panic in debug.
|
||||
debug_assert!(
|
||||
false,
|
||||
"PMActor {}: Free out of range (idx={}, order={}, total={})",
|
||||
self.actor_id,
|
||||
local_frame_idx,
|
||||
order,
|
||||
self.buddy.total_pages()
|
||||
);
|
||||
return None;
|
||||
}
|
||||
|
||||
self.buddy.free(local_frame_idx, order);
|
||||
|
||||
None
|
||||
}
|
||||
|
||||
fn handle_carve(
|
||||
&mut self,
|
||||
offset_pages: usize,
|
||||
size_pages: usize,
|
||||
channel_id: u16,
|
||||
) -> Option<PMResponse> {
|
||||
let range_pages = self.buddy.total_pages();
|
||||
if offset_pages >= range_pages
|
||||
|| size_pages == 0
|
||||
|| offset_pages.saturating_add(size_pages) > range_pages
|
||||
{
|
||||
return Some(PMResponse {
|
||||
channel_id,
|
||||
result: PMResult::OutOfMemory { size_pages },
|
||||
});
|
||||
}
|
||||
|
||||
let phys = PhysAddr(self.managed_range.0.0 + offset_pages as u64 * PAGE_SIZE);
|
||||
let cap = Capability {
|
||||
object: CapObject::Memory { phys, size_pages },
|
||||
rights: CapRights::READ | CapRights::WRITE | CapRights::GRANT,
|
||||
relation: Relation::Strong,
|
||||
token_sig: self.actor_id ^ offset_pages as u64,
|
||||
};
|
||||
|
||||
Some(PMResponse {
|
||||
channel_id,
|
||||
result: PMResult::Carved { cap },
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
194
kernel/src/mem/pm_router.rs
Normal file
194
kernel/src/mem/pm_router.rs
Normal file
@@ -0,0 +1,194 @@
|
||||
use core::sync::atomic::{AtomicU16, AtomicU8, AtomicBool, Ordering};
|
||||
use core::cell::UnsafeCell;
|
||||
use alloc::vec::Vec;
|
||||
use alloc::boxed::Box;
|
||||
|
||||
use crate::mem::pm_manages::{PMActor, PMRequest, PMResponse, PMResult};
|
||||
|
||||
const CHANNEL_COUNT: usize = 65536;
|
||||
const STATE_FREE: u8 = 0;
|
||||
const STATE_PENDING: u8 = 1;
|
||||
const STATE_READY: u8 = 2;
|
||||
|
||||
#[repr(align(64))]
|
||||
pub struct Channel {
|
||||
state: AtomicU8,
|
||||
next_free: AtomicU16,
|
||||
result: UnsafeCell<Option<PMResult>>,
|
||||
}
|
||||
|
||||
unsafe impl Send for Channel {}
|
||||
unsafe impl Sync for Channel {}
|
||||
|
||||
pub struct PMRouter {
|
||||
channels: Box<[Channel]>,
|
||||
free_head: AtomicU16,
|
||||
}
|
||||
|
||||
struct GlobalRouter {
|
||||
is_ready: AtomicBool,
|
||||
inner: UnsafeCell<Option<PMRouter>>,
|
||||
}
|
||||
|
||||
unsafe impl Sync for GlobalRouter {}
|
||||
unsafe impl Send for GlobalRouter {}
|
||||
|
||||
static ROUTER: GlobalRouter = GlobalRouter {
|
||||
is_ready: AtomicBool::new(false),
|
||||
inner: UnsafeCell::new(None),
|
||||
};
|
||||
|
||||
pub fn init() {
|
||||
if ROUTER.is_ready.load(Ordering::Acquire) {
|
||||
panic!("PMRouter is already initialized!");
|
||||
}
|
||||
|
||||
let mut channels = Vec::with_capacity(CHANNEL_COUNT);
|
||||
|
||||
for i in 0..CHANNEL_COUNT {
|
||||
channels.push(Channel {
|
||||
state: AtomicU8::new(STATE_FREE),
|
||||
next_free: AtomicU16::new((i + 1) as u16),
|
||||
result: UnsafeCell::new(None),
|
||||
});
|
||||
}
|
||||
|
||||
channels[CHANNEL_COUNT - 1].next_free.store(0, Ordering::Relaxed);
|
||||
|
||||
let router = PMRouter {
|
||||
channels: channels.into_boxed_slice(),
|
||||
free_head: AtomicU16::new(1),
|
||||
};
|
||||
|
||||
unsafe {
|
||||
*ROUTER.inner.get() = Some(router);
|
||||
}
|
||||
|
||||
ROUTER.is_ready.store(true, Ordering::Release);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn get_router() -> &'static PMRouter {
|
||||
// INVARIANT: `is_ready` is set once in `init()` and never cleared.
|
||||
// The Acquire load on `is_ready` synchronises-with the Release store in
|
||||
// `init()`, making the `Option::Some` write visible. Because nothing
|
||||
// ever writes `None` or clears `is_ready`, the TOCTOU window between
|
||||
// the load and the `unwrap_unchecked` is safe.
|
||||
if ROUTER.is_ready.load(Ordering::Acquire) {
|
||||
unsafe {
|
||||
(*ROUTER.inner.get()).as_ref().unwrap_unchecked()
|
||||
}
|
||||
} else {
|
||||
#[cold]
|
||||
fn not_initialized() -> ! {
|
||||
panic!("FATAL: PMRouter is accessed before initialization!")
|
||||
}
|
||||
not_initialized()
|
||||
}
|
||||
}
|
||||
|
||||
impl PMRouter {
|
||||
pub fn alloc_channel(&self) -> Option<u16> {
|
||||
let mut head = self.free_head.load(Ordering::Acquire);
|
||||
loop {
|
||||
if head == 0 {
|
||||
return None;
|
||||
}
|
||||
|
||||
let next = self.channels[head as usize].next_free.load(Ordering::Relaxed);
|
||||
|
||||
match self.free_head.compare_exchange_weak(
|
||||
head,
|
||||
next,
|
||||
Ordering::AcqRel,
|
||||
Ordering::Acquire,
|
||||
) {
|
||||
Ok(_) => {
|
||||
self.channels[head as usize].state.store(STATE_PENDING, Ordering::Release);
|
||||
return Some(head);
|
||||
}
|
||||
Err(new_head) => head = new_head,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn route_responses(&self, responses: Vec<PMResponse>) {
|
||||
for resp in responses {
|
||||
if resp.channel_id == 0 {
|
||||
continue;
|
||||
}
|
||||
|
||||
let idx = resp.channel_id as usize;
|
||||
|
||||
if idx >= CHANNEL_COUNT {
|
||||
panic!("PMRouter: Received response for out-of-bounds channel_id: {}", idx);
|
||||
}
|
||||
|
||||
let channel = &self.channels[idx];
|
||||
|
||||
unsafe {
|
||||
*channel.result.get() = Some(resp.result);
|
||||
}
|
||||
|
||||
channel.state.store(STATE_READY, Ordering::Release);
|
||||
|
||||
// Когда в будущем реализуешь Focus Mode, здесь нужно вызывать сигнал пробуждения конкретного процесса/потока (wake_up(thread_id)).
|
||||
}
|
||||
}
|
||||
|
||||
pub fn wait_for_response(&self, id: u16) -> PMResult {
|
||||
let channel = &self.channels[id as usize];
|
||||
|
||||
while channel.state.load(Ordering::Acquire) != STATE_READY {
|
||||
// HLT-ожидание вместо busy-wait: CPU останавливается до ближайшего
|
||||
// прерывания (timer tick, IPI от route_responses и т.д.).
|
||||
// Когда появится планировщик — заменить на yield_to_actor().
|
||||
unsafe { core::arch::asm!("hlt", options(nomem, nostack, preserves_flags)); }
|
||||
}
|
||||
|
||||
let result = unsafe {
|
||||
(*channel.result.get()).take().expect("PMRouter: Data missing on READY state")
|
||||
};
|
||||
|
||||
channel.state.store(STATE_FREE, Ordering::Release);
|
||||
|
||||
let mut head = self.free_head.load(Ordering::Relaxed);
|
||||
loop {
|
||||
channel.next_free.store(head, Ordering::Relaxed);
|
||||
match self.free_head.compare_exchange_weak(
|
||||
head,
|
||||
id,
|
||||
Ordering::Release,
|
||||
Ordering::Relaxed,
|
||||
) {
|
||||
Ok(_) => break,
|
||||
Err(new_head) => head = new_head,
|
||||
}
|
||||
}
|
||||
|
||||
result
|
||||
}
|
||||
}
|
||||
|
||||
pub fn request_and_wait<F>(actor: &PMActor, req_builder: F) -> PMResult
|
||||
where
|
||||
F: FnOnce(u16) -> PMRequest
|
||||
{
|
||||
let router = get_router();
|
||||
let channel_id = router.alloc_channel().expect("FATAL: Out of PM routing channels");
|
||||
|
||||
let req = req_builder(channel_id);
|
||||
|
||||
actor.submit_request(req).expect("FATAL: PMActor inbox is full");
|
||||
|
||||
// Временно вручную прокручиваем сообщения актёра (если мы пока работаем в 1 потоке).
|
||||
// Когда актёры переедут на отдельные ядра/треды, эту строчку нужно будет убрать,
|
||||
// так как актёр сам будет вызывать process_messages в бесконечном цикле.
|
||||
// router.route_responses(actor.process_messages()); // Включать только при тестировании в single-core!
|
||||
|
||||
router.wait_for_response(channel_id)
|
||||
}
|
||||
|
||||
pub fn dispatch(responses: Vec<PMResponse>) {
|
||||
get_router().route_responses(responses);
|
||||
}
|
||||
@@ -1,46 +1,87 @@
|
||||
use crate::mem::address::PhysAddr;
|
||||
use crate::mem::address::{PhysAddr, get_hhdm};
|
||||
use crate::mem::allocator::Locked;
|
||||
|
||||
pub const PAGE_SIZE: u64 = 4096;
|
||||
|
||||
pub struct BitmapPMM {
|
||||
bitmap: &'static mut [u8],
|
||||
l1_bitmap: &'static mut [u64],
|
||||
ref_counts: &'static mut [u16],
|
||||
total_pages: usize,
|
||||
used_pages: usize,
|
||||
last_idx: usize,
|
||||
last_word: usize,
|
||||
}
|
||||
|
||||
pub static PMM: Locked<Option<BitmapPMM>> = Locked::new(None);
|
||||
|
||||
impl BitmapPMM {
|
||||
unsafe fn read_word(&self, word_idx: usize) -> u64 {
|
||||
let byte_off = word_idx * 8;
|
||||
unsafe { (self.bitmap.as_ptr().add(byte_off) as *const u64).read() }
|
||||
}
|
||||
|
||||
fn l1_update_word(&mut self, word_idx: usize) {
|
||||
let word = unsafe { self.read_word(word_idx) };
|
||||
let l1_idx = word_idx / 64;
|
||||
let l1_bit = word_idx % 64;
|
||||
if word != !0u64 {
|
||||
self.l1_bitmap[l1_idx] |= 1 << l1_bit;
|
||||
} else {
|
||||
self.l1_bitmap[l1_idx] &= !(1 << l1_bit);
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub fn used_pages(&self) -> usize { self.used_pages }
|
||||
#[allow(dead_code)]
|
||||
pub fn total_pages(&self) -> usize { self.total_pages }
|
||||
#[allow(dead_code)]
|
||||
pub fn free_pages(&self) -> usize { self.total_pages.saturating_sub(self.used_pages) }
|
||||
|
||||
pub unsafe fn init(mmap: &limine::response::MemoryMapResponse, hhdm_offset: u64) {
|
||||
pub unsafe fn init(mmap: &limine::response::MemoryMapResponse) {
|
||||
let max_addr = mmap.entries().iter()
|
||||
.map(|e| e.base + e.length)
|
||||
.max()
|
||||
.unwrap_or(0);
|
||||
|
||||
let total_pages = (max_addr / PAGE_SIZE) as usize;
|
||||
|
||||
let bitmap_size = total_pages.div_ceil(8);
|
||||
let ref_counts_size = total_pages * core::mem::size_of::<u16>();
|
||||
let num_words = total_pages.div_ceil(64);
|
||||
let l1_u64_count = num_words.div_ceil(64);
|
||||
let l1_byte_size = l1_u64_count * 8;
|
||||
let total_meta_size = bitmap_size + ref_counts_size + l1_byte_size;
|
||||
|
||||
let bitmap_phys_addr = mmap.entries().iter()
|
||||
.find(|e| e.entry_type == limine::memory_map::EntryType::USABLE && e.length >= bitmap_size as u64)
|
||||
let meta_phys = mmap.entries().iter()
|
||||
.find(|e| {
|
||||
e.entry_type == limine::memory_map::EntryType::USABLE
|
||||
&& e.length >= total_meta_size as u64
|
||||
})
|
||||
.map(|e| e.base)
|
||||
.expect("PMM: Insufficient memory for bitmap");
|
||||
.expect("PMM: no usable region large enough for metadata");
|
||||
|
||||
let bitmap_virt_ptr = (bitmap_phys_addr + hhdm_offset) as *mut u8;
|
||||
let bitmap_slice = unsafe { core::slice::from_raw_parts_mut(bitmap_virt_ptr, bitmap_size) };
|
||||
bitmap_slice.fill(0xFF);
|
||||
let hhdm = get_hhdm();
|
||||
let bitmap_ptr = (meta_phys + hhdm) as *mut u8;
|
||||
let ref_counts_ptr = (meta_phys + hhdm + bitmap_size as u64) as *mut u16;
|
||||
let l1_ptr = (meta_phys + hhdm + bitmap_size as u64 + ref_counts_size as u64) as *mut u64;
|
||||
|
||||
let bitmap = unsafe { core::slice::from_raw_parts_mut(bitmap_ptr, bitmap_size) };
|
||||
bitmap.fill(0xFF);
|
||||
|
||||
let ref_counts = unsafe { core::slice::from_raw_parts_mut(ref_counts_ptr, total_pages) };
|
||||
ref_counts.fill(1);
|
||||
|
||||
let l1_bitmap = unsafe { core::slice::from_raw_parts_mut(l1_ptr, l1_u64_count) };
|
||||
l1_bitmap.fill(0);
|
||||
|
||||
let mut pmm = Self {
|
||||
bitmap: bitmap_slice,
|
||||
bitmap,
|
||||
l1_bitmap,
|
||||
ref_counts,
|
||||
total_pages,
|
||||
used_pages: total_pages,
|
||||
last_idx: 0,
|
||||
last_word: 0,
|
||||
};
|
||||
|
||||
for entry in mmap.entries() {
|
||||
@@ -51,7 +92,8 @@ impl BitmapPMM {
|
||||
}
|
||||
}
|
||||
|
||||
for addr in (bitmap_phys_addr..bitmap_phys_addr + bitmap_size as u64).step_by(PAGE_SIZE as usize) {
|
||||
let meta_end = (meta_phys + total_meta_size as u64 + PAGE_SIZE - 1) & !(PAGE_SIZE - 1);
|
||||
for addr in (meta_phys..meta_end).step_by(PAGE_SIZE as usize) {
|
||||
pmm.lock_frame(PhysAddr(addr));
|
||||
}
|
||||
|
||||
@@ -62,66 +104,156 @@ impl BitmapPMM {
|
||||
|
||||
pub fn free_frame(&mut self, phys_addr: PhysAddr) {
|
||||
let idx = (phys_addr.0 / PAGE_SIZE) as usize;
|
||||
if idx < self.total_pages {
|
||||
let byte_idx = idx / 8;
|
||||
let bit_idx = idx % 8;
|
||||
if (self.bitmap[byte_idx] & (1 << bit_idx)) != 0 {
|
||||
self.bitmap[byte_idx] &= !(1 << bit_idx);
|
||||
if idx >= self.total_pages { return; }
|
||||
|
||||
let byte = idx / 8;
|
||||
let bit = idx % 8;
|
||||
|
||||
if self.bitmap[byte] & (1 << bit) != 0 {
|
||||
self.ref_counts[idx] = self.ref_counts[idx].saturating_sub(1);
|
||||
|
||||
if self.ref_counts[idx] == 0 {
|
||||
self.bitmap[byte] &= !(1 << bit);
|
||||
self.used_pages -= 1;
|
||||
|
||||
let word_idx = idx / 64;
|
||||
self.l1_update_word(word_idx);
|
||||
if word_idx < self.last_word { self.last_word = word_idx; }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn lock_frame(&mut self, phys_addr: PhysAddr) {
|
||||
let idx = (phys_addr.0 / PAGE_SIZE) as usize;
|
||||
if idx < self.total_pages {
|
||||
let byte_idx = idx / 8;
|
||||
let bit_idx = idx % 8;
|
||||
if (self.bitmap[byte_idx] & (1 << bit_idx)) == 0 {
|
||||
self.bitmap[byte_idx] |= 1 << bit_idx;
|
||||
if idx >= self.total_pages { return; }
|
||||
|
||||
let byte = idx / 8;
|
||||
let bit = idx % 8;
|
||||
|
||||
if self.bitmap[byte] & (1 << bit) == 0 {
|
||||
self.bitmap[byte] |= 1 << bit;
|
||||
self.ref_counts[idx] = 1;
|
||||
self.used_pages += 1;
|
||||
} else if self.ref_counts[idx] == 0 {
|
||||
self.ref_counts[idx] = 1;
|
||||
}
|
||||
|
||||
let word_idx = idx / 64;
|
||||
self.l1_update_word(word_idx);
|
||||
}
|
||||
|
||||
pub fn inc_ref_frame(&mut self, phys_addr: PhysAddr) {
|
||||
let idx = (phys_addr.0 / PAGE_SIZE) as usize;
|
||||
if idx >= self.total_pages { return; }
|
||||
|
||||
let byte = idx / 8;
|
||||
let bit = idx % 8;
|
||||
|
||||
if self.bitmap[byte] & (1 << bit) != 0 {
|
||||
self.ref_counts[idx] = self.ref_counts[idx].saturating_add(1);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn alloc_frame(&mut self) -> Option<PhysAddr> {
|
||||
let start_byte = self.last_idx / 8;
|
||||
for i in start_byte..(self.bitmap.len()) {
|
||||
if self.bitmap[i] != 0xFF {
|
||||
for bit in 0..8 {
|
||||
let idx = i * 8 + bit;
|
||||
if idx >= self.total_pages { return None; }
|
||||
let l1_len = self.l1_bitmap.len();
|
||||
|
||||
let addr = PhysAddr(idx as u64 * PAGE_SIZE);
|
||||
if !self.is_locked(addr) {
|
||||
self.lock_frame(addr);
|
||||
self.last_idx = idx;
|
||||
return Some(addr);
|
||||
}
|
||||
}
|
||||
for pass in 0..2usize {
|
||||
let (from, to) = if pass == 0 {
|
||||
(self.last_word / 64, l1_len)
|
||||
} else {
|
||||
(0, self.last_word / 64)
|
||||
};
|
||||
|
||||
for l1_idx in from..to {
|
||||
let l1_word = self.l1_bitmap[l1_idx];
|
||||
if l1_word == 0 { continue; }
|
||||
|
||||
let word_offset = l1_word.trailing_zeros() as usize;
|
||||
let word_idx = l1_idx * 64 + word_offset;
|
||||
|
||||
let word = unsafe { self.read_word(word_idx) };
|
||||
if word == !0u64 {
|
||||
self.l1_bitmap[l1_idx] &= !(1 << word_offset);
|
||||
continue;
|
||||
}
|
||||
|
||||
let free_bit = (!word).trailing_zeros() as usize;
|
||||
let page_idx = word_idx * 64 + free_bit;
|
||||
if page_idx >= self.total_pages { continue; }
|
||||
|
||||
let byte = page_idx / 8;
|
||||
let bit = page_idx % 8;
|
||||
|
||||
self.bitmap[byte] |= 1 << bit;
|
||||
self.ref_counts[page_idx] = 1;
|
||||
self.used_pages += 1;
|
||||
self.last_word = word_idx;
|
||||
|
||||
if unsafe { self.read_word(word_idx) } == !0u64 {
|
||||
self.l1_bitmap[l1_idx] &= !(1 << word_offset);
|
||||
}
|
||||
|
||||
return Some(PhysAddr(page_idx as u64 * PAGE_SIZE));
|
||||
}
|
||||
}
|
||||
|
||||
None
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn is_locked(&self, addr: PhysAddr) -> bool {
|
||||
let idx = (addr.0 / PAGE_SIZE) as usize;
|
||||
(self.bitmap[idx / 8] & (1 << (idx % 8))) != 0
|
||||
pub fn alloc_contiguous(&mut self, count: usize) -> Option<PhysAddr> {
|
||||
if count == 0 { return None; }
|
||||
|
||||
let mut run_start = 0usize;
|
||||
let mut run_len = 0usize;
|
||||
|
||||
for page_idx in 0..self.total_pages {
|
||||
let byte = page_idx / 8;
|
||||
let bit = page_idx % 8;
|
||||
|
||||
if self.bitmap[byte] & (1 << bit) == 0 {
|
||||
if run_len == 0 { run_start = page_idx; }
|
||||
run_len += 1;
|
||||
|
||||
if run_len == count {
|
||||
for i in run_start..run_start + count {
|
||||
self.bitmap[i / 8] |= 1 << (i % 8);
|
||||
self.ref_counts[i] = 1;
|
||||
let word_idx = i / 64;
|
||||
self.l1_update_word(word_idx);
|
||||
}
|
||||
self.used_pages += count;
|
||||
self.last_word = run_start / 64;
|
||||
return Some(PhysAddr(run_start as u64 * PAGE_SIZE));
|
||||
}
|
||||
} else {
|
||||
run_len = 0;
|
||||
}
|
||||
}
|
||||
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
pub fn alloc_frame() -> Option<PhysAddr> {
|
||||
PMM.lock().as_mut()?.alloc_frame()
|
||||
}
|
||||
|
||||
pub fn alloc_contiguous(count: usize) -> Option<PhysAddr> {
|
||||
PMM.lock().as_mut()?.alloc_contiguous(count)
|
||||
}
|
||||
|
||||
pub fn free_frame(addr: PhysAddr) {
|
||||
if let Some(pmm) = PMM.lock().as_mut() {
|
||||
pmm.free_frame(addr);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn inc_ref_frame(addr: PhysAddr) {
|
||||
if let Some(pmm) = PMM.lock().as_mut() {
|
||||
pmm.inc_ref_frame(addr);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn get_stats() -> (usize, usize) {
|
||||
if let Some(pmm) = PMM.lock().as_ref() {
|
||||
(pmm.used_pages(), pmm.total_pages())
|
||||
|
||||
816
kernel/src/mem/vmm.rs
Normal file
816
kernel/src/mem/vmm.rs
Normal file
@@ -0,0 +1,816 @@
|
||||
//! # Virtual Memory Manager (`src/mem/vmm.rs`)
|
||||
//!
|
||||
//! Manages address spaces (PML4 + VMA list) and provides the bridge between
|
||||
//! the capability system and the hardware MMU.
|
||||
//!
|
||||
//! ## Changes vs. previous version
|
||||
//!
|
||||
//! 1. **ASID recycling** — replaced the monotonic `AtomicU16` with a 4096-bit
|
||||
//! bitmap allocator (`AsidAllocator`). `AddressSpace::drop` returns the ASID
|
||||
//! to the pool, making it available for future processes. The bitmap is
|
||||
//! `const`-initializable (no heap), so no boot-ordering issues.
|
||||
//!
|
||||
//! 2. **INVPCID CPUID guard** — `init_cpu_features()` must be called once during
|
||||
//! boot (before any `AddressSpace::activate()` or TLB flush). It checks
|
||||
//! CPUID.07H:EBX[10] and sets `INVPCID_SUPPORTED`. `tlb_flush_asid` falls
|
||||
//! back to a full CR3 reload on CPUs that lack the instruction.
|
||||
//!
|
||||
//! 3. **Zero-copy shared mappings** — new `AddressSpace::map_shared()` maps a
|
||||
//! physical range owned by another capability into this address space without
|
||||
//! copying the frames. `VmaBacking::Shared` stores the `phys_base` so the
|
||||
//! physical address is retrievable at any time via `translate()` on the PT.
|
||||
//! Frames are *not* freed on unmap — ownership stays with the issuing actor.
|
||||
//!
|
||||
//! All other behaviour (lazy demand paging, revocation, TLB management) is
|
||||
//! unchanged from the previous version.
|
||||
|
||||
#![allow(dead_code)]
|
||||
|
||||
use alloc::vec::Vec;
|
||||
use core::sync::atomic::{AtomicBool, AtomicU32, Ordering, AtomicU64, AtomicU16};
|
||||
|
||||
use crate::mem::address::{PhysAddr, VirtAddr};
|
||||
use crate::mem::allocator::Locked;
|
||||
use crate::mem::paging::{PageTable, PageTableFlags};
|
||||
use crate::mem::pmm;
|
||||
use crate::events::MMU_REVOCATION_QUEUE;
|
||||
|
||||
|
||||
use core::hint::spin_loop;
|
||||
|
||||
extern crate alloc;
|
||||
|
||||
pub static ACTIVE_CPUS_MASK: AtomicU64 = AtomicU64::new(1);
|
||||
|
||||
static SHOOTDOWN_LOCK: Locked<()> = Locked::new(());
|
||||
static SHOOTDOWN_ASID: AtomicU16 = AtomicU16::new(0);
|
||||
static SHOOTDOWN_ACK: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
#[inline]
|
||||
fn local_tlb_flush_asid(asid: u16) {
|
||||
if INVPCID_SUPPORTED.load(Ordering::Relaxed) {
|
||||
#[repr(C, packed)]
|
||||
struct InvpcidDesc { pcid: u64, addr: u64 }
|
||||
|
||||
let desc = InvpcidDesc { pcid: asid as u64, addr: 0 };
|
||||
unsafe {
|
||||
core::arch::asm!(
|
||||
"invpcid {ty}, [{desc}]",
|
||||
ty = in(reg) 1u64,
|
||||
desc = in(reg) &desc,
|
||||
options(nostack, preserves_flags),
|
||||
);
|
||||
}
|
||||
} else {
|
||||
tlb_flush_all();
|
||||
}
|
||||
}
|
||||
|
||||
// Error type
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum VmError {
|
||||
OutOfMemory,
|
||||
RegionOverlap,
|
||||
RegionNotFound,
|
||||
InvalidAlignment,
|
||||
InvalidRange,
|
||||
PermissionDenied,
|
||||
UnexpectedFault,
|
||||
NonCanonical,
|
||||
AsidExhausted,
|
||||
}
|
||||
|
||||
impl core::fmt::Display for VmError {
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
f.write_str(match self {
|
||||
Self::OutOfMemory => "physical memory exhausted",
|
||||
Self::RegionOverlap => "virtual address range overlaps existing VMA",
|
||||
Self::RegionNotFound => "no VMA at address",
|
||||
Self::InvalidAlignment => "address/size not page-aligned",
|
||||
Self::InvalidRange => "zero-size or overflowing range",
|
||||
Self::PermissionDenied => "VMA permission denied",
|
||||
Self::UnexpectedFault => "page fault in non-lazy region",
|
||||
Self::NonCanonical => "non-canonical virtual address",
|
||||
Self::AsidExhausted => "ASID/PCID pool exhausted",
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// VMA flags
|
||||
|
||||
bitflags::bitflags! {
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct VmaFlags: u32 {
|
||||
const READ = 1 << 0;
|
||||
const WRITE = 1 << 1;
|
||||
const EXEC = 1 << 2;
|
||||
const USER = 1 << 3;
|
||||
const LAZY = 1 << 4;
|
||||
const SHARED = 1 << 5;
|
||||
const PINNED = 1 << 6;
|
||||
const NOCACHE = 1 << 7;
|
||||
const MMIO = 1 << 8;
|
||||
const COW = 1 << 9;
|
||||
}
|
||||
}
|
||||
|
||||
impl VmaFlags {
|
||||
#[inline]
|
||||
pub fn to_page_flags(self) -> PageTableFlags {
|
||||
let mut f = PageTableFlags::PRESENT;
|
||||
if self.contains(Self::WRITE) { f |= PageTableFlags::WRITABLE; }
|
||||
if self.contains(Self::USER) { f |= PageTableFlags::USER; }
|
||||
if !self.contains(Self::EXEC) { f |= PageTableFlags::NO_EXECUTE; }
|
||||
if self.contains(Self::NOCACHE) || self.contains(Self::MMIO) {
|
||||
f |= PageTableFlags::NO_CACHE | PageTableFlags::WRITE_THROUGH;
|
||||
}
|
||||
if self.contains(Self::COW) { f |= PageTableFlags::COW; }
|
||||
f
|
||||
}
|
||||
}
|
||||
|
||||
// VMA backing
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum VmaBacking {
|
||||
Anonymous(Vec<Option<PhysAddr>>),
|
||||
Physical(PhysAddr),
|
||||
Shared {
|
||||
owner_cap: u64,
|
||||
phys_base: PhysAddr,
|
||||
},
|
||||
}
|
||||
|
||||
impl VmaBacking {
|
||||
#[inline]
|
||||
fn phys_for_page(&self, page_idx: usize) -> Option<PhysAddr> {
|
||||
match self {
|
||||
Self::Anonymous(frames) => frames.get(page_idx).copied().flatten(),
|
||||
Self::Physical(base) => Some(PhysAddr(base.0 + page_idx as u64 * 4096)),
|
||||
Self::Shared { phys_base, .. } => {
|
||||
Some(PhysAddr(phys_base.0 + page_idx as u64 * 4096))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether this backing owns its frames (should the PMM free them on unmap).
|
||||
#[inline]
|
||||
fn owns_frames(&self) -> bool {
|
||||
matches!(self, Self::Anonymous(_))
|
||||
}
|
||||
}
|
||||
|
||||
// VMA region
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct VmaRegion {
|
||||
pub virt_start: VirtAddr,
|
||||
pub virt_end: VirtAddr,
|
||||
pub flags: VmaFlags,
|
||||
pub cap_token: u64,
|
||||
pub backing: VmaBacking,
|
||||
}
|
||||
|
||||
impl VmaRegion {
|
||||
#[inline] pub fn size(&self) -> u64 { self.virt_end.0 - self.virt_start.0 }
|
||||
#[inline] pub fn pages(&self) -> usize { (self.size() / 4096) as usize }
|
||||
#[inline] pub fn contains(&self, a: VirtAddr) -> bool {
|
||||
a.0 >= self.virt_start.0 && a.0 < self.virt_end.0
|
||||
}
|
||||
}
|
||||
|
||||
// ASID / PCID allocator (bitmap-based, const-initializable, O(1) amortised)
|
||||
|
||||
/// x86-64 PCIDs: 0 (kernel, no PCID tagging) and 4095 (reserved by spec).
|
||||
/// Valid user ASIDs: 1 – 4094 inclusive.
|
||||
///
|
||||
/// Layout: `bitmap[i]` bit `j` represents ASID `i * 32 + j`.
|
||||
/// Bit set = in use.
|
||||
struct AsidAllocator {
|
||||
/// 4096 bits in 128 × u32 words.
|
||||
bitmap: [u32; 128],
|
||||
/// Start-of-next-search hint (in ASID units, not word units).
|
||||
next_hint: u16,
|
||||
}
|
||||
|
||||
impl AsidAllocator {
|
||||
const fn new() -> Self {
|
||||
let mut bm = [0u32; 128];
|
||||
// Mark ASID 0 (kernel) as permanently in-use.
|
||||
bm[0] |= 1u32;
|
||||
// Mark ASID 4095 (reserved by Intel spec) as permanently in-use.
|
||||
// 4095 / 32 = 127, bit 31.
|
||||
bm[127] |= 1u32 << 31;
|
||||
Self { bitmap: bm, next_hint: 1 }
|
||||
}
|
||||
|
||||
/// Allocate the lowest free ASID in [1, 4094]. O(1) amortised with hint.
|
||||
fn alloc(&mut self) -> Option<u16> {
|
||||
// Two-pass: start from hint, wrap around if necessary.
|
||||
for pass in 0..2usize {
|
||||
let start = if pass == 0 { self.next_hint as usize } else { 1 };
|
||||
let end = if pass == 0 { 4095usize } else { self.next_hint as usize };
|
||||
|
||||
let mut asid = start;
|
||||
while asid < end {
|
||||
let word = asid / 32;
|
||||
let bit = asid % 32;
|
||||
if self.bitmap[word] & (1u32 << bit) == 0 {
|
||||
self.bitmap[word] |= 1u32 << bit;
|
||||
self.next_hint = ((asid + 1) as u16).min(4094);
|
||||
return Some(asid as u16);
|
||||
}
|
||||
// Skip fully-used 32-bit words for amortised O(1).
|
||||
if self.bitmap[word] == u32::MAX {
|
||||
asid = (word + 1) * 32;
|
||||
} else {
|
||||
asid += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Return an ASID to the pool.
|
||||
fn free(&mut self, asid: u16) {
|
||||
if asid == 0 || asid >= 4095 {
|
||||
return;
|
||||
}
|
||||
let word = asid as usize / 32;
|
||||
let bit = asid as usize % 32;
|
||||
// Clear the bit. Idempotent: double-free is safe but wasteful.
|
||||
self.bitmap[word] &= !(1u32 << bit);
|
||||
// Pull hint back so reuse happens quickly.
|
||||
if asid < self.next_hint {
|
||||
self.next_hint = asid;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static ASID_ALLOC: Locked<AsidAllocator> = Locked::new(AsidAllocator::new());
|
||||
|
||||
#[inline]
|
||||
fn alloc_asid() -> Result<u16, VmError> {
|
||||
ASID_ALLOC.lock().alloc().ok_or(VmError::AsidExhausted)
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn free_asid(asid: u16) {
|
||||
ASID_ALLOC.lock().free(asid);
|
||||
}
|
||||
|
||||
// CPU feature detection (call once during boot, before first activate())
|
||||
|
||||
/// Set to `true` at boot if CPUID.07H:EBX[10] = 1 (INVPCID supported).
|
||||
static INVPCID_SUPPORTED: AtomicBool = AtomicBool::new(false);
|
||||
|
||||
/// Detect and record the availability of INVPCID.
|
||||
///
|
||||
/// Must be called **once** during kernel init (before `init_kernel_space` or
|
||||
/// any `AddressSpace::activate`). Idempotent if called more than once.
|
||||
pub fn init_cpu_features() {
|
||||
// CPUID leaf 7, sub-leaf 0 — INVPCID is EBX[10].
|
||||
//
|
||||
// LLVM permanently reserves RBX for its own use in inline asm, so we
|
||||
// cannot name it as an operand directly. The standard workaround is to
|
||||
// spill RBX around the CPUID instruction ourselves.
|
||||
let ebx: u32;
|
||||
unsafe {
|
||||
core::arch::asm!(
|
||||
// Spill rbx (LLVM's reserved reg) to a caller-saved scratch reg.
|
||||
"mov {tmp:r}, rbx",
|
||||
"mov eax, 7",
|
||||
"xor ecx, ecx",
|
||||
"cpuid",
|
||||
// Move the result out before restoring rbx.
|
||||
"mov {out:e}, ebx",
|
||||
"mov rbx, {tmp:r}",
|
||||
tmp = out(reg) _, // any scratch register chosen by LLVM
|
||||
out = out(reg) ebx,
|
||||
out("eax") _,
|
||||
out("ecx") _,
|
||||
out("edx") _,
|
||||
options(nostack, preserves_flags),
|
||||
);
|
||||
}
|
||||
let supported = (ebx >> 10) & 1 == 1;
|
||||
INVPCID_SUPPORTED.store(supported, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
// Address space
|
||||
|
||||
pub struct AddressSpace {
|
||||
/// Hardware PCID written to CR3 bits 11:0.
|
||||
pub asid: u16,
|
||||
/// Physical address of the PML4 page.
|
||||
pub pml4_phys: PhysAddr,
|
||||
/// Sorted (by virt_start), non-overlapping VMA list.
|
||||
regions: Vec<VmaRegion>,
|
||||
}
|
||||
|
||||
impl AddressSpace {
|
||||
#[inline]
|
||||
unsafe fn pml4_raw(&self) -> *mut PageTable {
|
||||
self.pml4_phys.to_virt().as_mut_ptr::<PageTable>()
|
||||
}
|
||||
|
||||
fn check_overlap(&self, start: VirtAddr, end: VirtAddr) -> Result<(), VmError> {
|
||||
for r in &self.regions {
|
||||
if r.virt_start.0 < end.0 && r.virt_end.0 > start.0 {
|
||||
return Err(VmError::RegionOverlap);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn clone_for_fork(&mut self, child_cap_token: u64) -> Result<Self, VmError> {
|
||||
let mut child = AddressSpace::new()?;
|
||||
let child_pml4 = unsafe { &mut *child.pml4_raw() };
|
||||
let parent_pml4 = unsafe { &mut *self.pml4_raw() };
|
||||
|
||||
for region in &mut self.regions {
|
||||
let mut child_region = VmaRegion {
|
||||
virt_start: region.virt_start,
|
||||
virt_end: region.virt_end,
|
||||
flags: region.flags,
|
||||
cap_token: child_cap_token,
|
||||
backing: match ®ion.backing {
|
||||
VmaBacking::Physical(base) => VmaBacking::Physical(*base),
|
||||
VmaBacking::Shared { owner_cap, phys_base } => VmaBacking::Shared { owner_cap: *owner_cap, phys_base: *phys_base },
|
||||
VmaBacking::Anonymous(frames) => {
|
||||
let mut new_frames = Vec::with_capacity(frames.len());
|
||||
new_frames.resize_with(frames.len(), || None);
|
||||
VmaBacking::Anonymous(new_frames)
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
match &mut region.backing {
|
||||
VmaBacking::Physical(base) => {
|
||||
child_pml4.map_region(region.virt_start, *base, region.size(), region.flags.to_page_flags());
|
||||
}
|
||||
VmaBacking::Shared { phys_base, .. } => {
|
||||
child_pml4.map_region(region.virt_start, *phys_base, region.size(), region.flags.to_page_flags());
|
||||
}
|
||||
VmaBacking::Anonymous(frames) => {
|
||||
let cow_needed = region.flags.contains(VmaFlags::WRITE);
|
||||
|
||||
if cow_needed {
|
||||
region.flags.insert(VmaFlags::COW);
|
||||
child_region.flags.insert(VmaFlags::COW);
|
||||
}
|
||||
|
||||
let VmaBacking::Anonymous(ref mut child_frames) = child_region.backing else { unreachable!() };
|
||||
|
||||
for (i, frame_opt) in frames.iter().enumerate() {
|
||||
if let Some(frame) = frame_opt {
|
||||
let virt = VirtAddr(region.virt_start.0 + i as u64 * 4096);
|
||||
|
||||
pmm::inc_ref_frame(*frame);
|
||||
|
||||
let mut page_flags = region.flags.to_page_flags();
|
||||
|
||||
if cow_needed {
|
||||
page_flags.remove(PageTableFlags::WRITABLE);
|
||||
page_flags.insert(PageTableFlags::COW);
|
||||
|
||||
let _ = parent_pml4.update_flags(virt, page_flags);
|
||||
}
|
||||
|
||||
child_pml4.map_page(virt, *frame, page_flags);
|
||||
child_frames[i] = Some(*frame);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
child.regions.push(child_region);
|
||||
}
|
||||
|
||||
tlb_flush_asid(self.asid);
|
||||
|
||||
Ok(child)
|
||||
}
|
||||
|
||||
|
||||
|
||||
fn insert_sorted(&mut self, region: VmaRegion) {
|
||||
let pos = self.regions
|
||||
.partition_point(|r| r.virt_start.0 < region.virt_start.0);
|
||||
self.regions.insert(pos, region);
|
||||
}
|
||||
|
||||
fn find_idx(&self, addr: VirtAddr) -> Option<usize> {
|
||||
let pos = self.regions.partition_point(|r| r.virt_end.0 <= addr.0);
|
||||
self.regions.get(pos).filter(|r| r.contains(addr)).map(|_| pos)
|
||||
}
|
||||
|
||||
/// Unmap pages + free frames (if owned) for one region.
|
||||
fn do_unmap(pml4: *mut PageTable, region: &VmaRegion) {
|
||||
let pml4 = unsafe { &mut *pml4 };
|
||||
let own = region.backing.owns_frames();
|
||||
for i in 0..region.pages() {
|
||||
let virt = VirtAddr(region.virt_start.0 + i as u64 * 4096);
|
||||
pml4.unmap_page(virt);
|
||||
if own {
|
||||
if let Some(frame) = region.backing.phys_for_page(i) {
|
||||
pmm::free_frame(frame);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn do_revoke_by_token(&mut self, cap_token: u64) {
|
||||
let pml4 = unsafe { self.pml4_raw() };
|
||||
|
||||
let indices: Vec<usize> = self.regions
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(_, r)| r.cap_token == cap_token && !r.flags.contains(VmaFlags::PINNED))
|
||||
.map(|(i, _)| i)
|
||||
.collect();
|
||||
|
||||
for idx in indices.into_iter().rev() {
|
||||
let region = self.regions.remove(idx);
|
||||
Self::do_unmap(pml4, ®ion);
|
||||
}
|
||||
}
|
||||
|
||||
/// Drain the MMU revocation queue and apply all pending token revocations.
|
||||
/// Called by the page-fault handler before acquiring the VMM lock to
|
||||
/// prevent deadlocks with the capability subsystem.
|
||||
pub fn process_pending_revocations(&mut self) {
|
||||
let mut needs_flush = false;
|
||||
while let Some(token) = MMU_REVOCATION_QUEUE.pop() {
|
||||
self.do_revoke_by_token(token);
|
||||
needs_flush = true;
|
||||
}
|
||||
if needs_flush {
|
||||
tlb_flush_asid(self.asid);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl AddressSpace {
|
||||
/// Allocate a fresh, empty address space with a zeroed PML4.
|
||||
pub fn new() -> Result<Self, VmError> {
|
||||
let pml4_phys = pmm::alloc_frame().ok_or(VmError::OutOfMemory)?;
|
||||
unsafe {
|
||||
core::ptr::write_bytes(pml4_phys.to_virt().as_mut_ptr::<u8>(), 0, 4096);
|
||||
}
|
||||
Ok(Self {
|
||||
asid: alloc_asid()?,
|
||||
pml4_phys,
|
||||
regions: Vec::new(),
|
||||
})
|
||||
}
|
||||
|
||||
/// Wrap an already-active PML4 (bootloader-provided kernel table).
|
||||
/// ASID 0 = kernel "no PCID tagging"; it is never returned to the pool.
|
||||
pub fn from_active(pml4_phys: PhysAddr, asid: u16) -> Self {
|
||||
Self { asid, pml4_phys, regions: Vec::new() }
|
||||
}
|
||||
|
||||
pub fn map_region(
|
||||
&mut self,
|
||||
virt: VirtAddr,
|
||||
phys: Option<PhysAddr>,
|
||||
size: u64,
|
||||
flags: VmaFlags,
|
||||
cap_token: u64,
|
||||
) -> Result<VirtAddr, VmError> {
|
||||
if size == 0 { return Err(VmError::InvalidRange); }
|
||||
if virt.0 & 0xFFF != 0 || size & 0xFFF != 0 {
|
||||
return Err(VmError::InvalidAlignment);
|
||||
}
|
||||
|
||||
let virt_end = VirtAddr(virt.0.checked_add(size).ok_or(VmError::InvalidRange)?);
|
||||
self.check_overlap(virt, virt_end)?;
|
||||
|
||||
let pml4 = unsafe { &mut *self.pml4_raw() };
|
||||
let page_flags = flags.to_page_flags();
|
||||
let page_count = (size / 4096) as usize;
|
||||
|
||||
let backing = match phys {
|
||||
Some(base) => {
|
||||
pml4.map_region(virt, base, size, page_flags);
|
||||
VmaBacking::Physical(base)
|
||||
}
|
||||
None if flags.contains(VmaFlags::LAZY) => {
|
||||
let mut frames = Vec::with_capacity(page_count);
|
||||
frames.resize_with(page_count, || None);
|
||||
VmaBacking::Anonymous(frames)
|
||||
}
|
||||
None => {
|
||||
let mut frames = Vec::with_capacity(page_count);
|
||||
for i in 0..page_count {
|
||||
let frame = pmm::alloc_frame().ok_or(VmError::OutOfMemory)?;
|
||||
unsafe {
|
||||
core::ptr::write_bytes(frame.to_virt().as_mut_ptr::<u8>(), 0, 4096);
|
||||
}
|
||||
pml4.map_page(
|
||||
VirtAddr(virt.0 + i as u64 * 4096),
|
||||
frame,
|
||||
page_flags,
|
||||
);
|
||||
frames.push(Some(frame));
|
||||
}
|
||||
VmaBacking::Anonymous(frames)
|
||||
}
|
||||
};
|
||||
|
||||
self.insert_sorted(VmaRegion { virt_start: virt, virt_end, flags, cap_token, backing });
|
||||
Ok(virt)
|
||||
}
|
||||
|
||||
/// Map `page_count` pages of physical memory owned by `owner_cap` into
|
||||
/// this address space at `virt`.
|
||||
///
|
||||
/// This is the primitive behind Elyz's zero-copy transfer: the physical
|
||||
/// frames stay in place; only the PTE entries (rights) move between address
|
||||
/// spaces. The frames are **not** freed when this mapping is unmapped —
|
||||
/// the owning actor's capability retains that responsibility.
|
||||
///
|
||||
/// # Revocation
|
||||
/// Pass `owner_cap` as `cap_token` in the VMA so that `revoke_by_token`
|
||||
/// finds and unmaps this shared view when the source capability is revoked.
|
||||
pub fn map_shared(
|
||||
&mut self,
|
||||
virt: VirtAddr,
|
||||
phys_base: PhysAddr,
|
||||
page_count: usize,
|
||||
flags: VmaFlags,
|
||||
owner_cap: u64,
|
||||
) -> Result<VirtAddr, VmError> {
|
||||
if page_count == 0 { return Err(VmError::InvalidRange); }
|
||||
if virt.0 & 0xFFF != 0 { return Err(VmError::InvalidAlignment); }
|
||||
|
||||
let size = page_count as u64 * 4096;
|
||||
let virt_end = VirtAddr(virt.0.checked_add(size).ok_or(VmError::InvalidRange)?);
|
||||
self.check_overlap(virt, virt_end)?;
|
||||
|
||||
let pml4 = unsafe { &mut *self.pml4_raw() };
|
||||
let page_flags = flags.to_page_flags();
|
||||
|
||||
// Eagerly map all pages — the physical addresses are already known.
|
||||
for i in 0..page_count {
|
||||
pml4.map_page(
|
||||
VirtAddr(virt.0 + i as u64 * 4096),
|
||||
PhysAddr(phys_base.0 + i as u64 * 4096),
|
||||
page_flags,
|
||||
);
|
||||
}
|
||||
|
||||
self.insert_sorted(VmaRegion {
|
||||
virt_start: virt,
|
||||
virt_end,
|
||||
flags: flags | VmaFlags::SHARED,
|
||||
cap_token: owner_cap,
|
||||
backing: VmaBacking::Shared { owner_cap, phys_base },
|
||||
});
|
||||
|
||||
Ok(virt)
|
||||
}
|
||||
|
||||
/// Handle a hardware page fault at `fault_addr`.
|
||||
///
|
||||
/// Returns `Ok(())` if the fault was a valid lazy demand-page (caller should
|
||||
/// `iretq` to retry). Returns `Err` for illegal accesses.
|
||||
pub fn handle_fault(&mut self, fault_addr: VirtAddr, write: bool) -> Result<(), VmError> {
|
||||
let idx = self.find_idx(fault_addr).ok_or(VmError::RegionNotFound)?;
|
||||
|
||||
let (virt_start, region_flags) = {
|
||||
let r = &self.regions[idx];
|
||||
(r.virt_start, r.flags)
|
||||
};
|
||||
|
||||
if write && !region_flags.contains(VmaFlags::WRITE) {
|
||||
return Err(VmError::PermissionDenied);
|
||||
}
|
||||
|
||||
let page_idx = ((fault_addr.0 - virt_start.0) / 4096) as usize;
|
||||
let page_virt = VirtAddr(virt_start.0 + page_idx as u64 * 4096);
|
||||
let pml4 = unsafe { &mut *self.pml4_raw() };
|
||||
|
||||
let current_pte_flags = pml4.get_flags(page_virt);
|
||||
let is_cow = current_pte_flags.map_or(false, |f| f.contains(PageTableFlags::COW));
|
||||
|
||||
if write && is_cow {
|
||||
let region = &mut self.regions[idx];
|
||||
let VmaBacking::Anonymous(ref mut frames) = region.backing else {
|
||||
return Err(VmError::UnexpectedFault);
|
||||
};
|
||||
|
||||
let old_frame = frames[page_idx].expect("COW fault on unmapped page");
|
||||
let new_frame = pmm::alloc_frame().ok_or(VmError::OutOfMemory)?;
|
||||
|
||||
debug_assert!(old_frame != new_frame, "COW: old and new frame are the same!");
|
||||
|
||||
unsafe {
|
||||
core::ptr::copy_nonoverlapping(
|
||||
old_frame.to_virt().as_ptr::<u8>(),
|
||||
new_frame.to_virt().as_mut_ptr::<u8>(),
|
||||
4096,
|
||||
);
|
||||
}
|
||||
|
||||
frames[page_idx] = Some(new_frame);
|
||||
|
||||
let mut target_flags = region.flags.to_page_flags();
|
||||
target_flags.remove(PageTableFlags::COW);
|
||||
target_flags.insert(PageTableFlags::WRITABLE);
|
||||
|
||||
pml4.map_page(page_virt, new_frame, target_flags);
|
||||
|
||||
pmm::free_frame(old_frame);
|
||||
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
if !region_flags.contains(VmaFlags::LAZY) {
|
||||
return Err(VmError::UnexpectedFault);
|
||||
}
|
||||
|
||||
let region = &mut self.regions[idx];
|
||||
let VmaBacking::Anonymous(ref mut frames) = region.backing else {
|
||||
return Err(VmError::RegionNotFound);
|
||||
};
|
||||
|
||||
if frames[page_idx].is_some() {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let frame = pmm::alloc_frame().ok_or(VmError::OutOfMemory)?;
|
||||
unsafe {
|
||||
core::ptr::write_bytes(frame.to_virt().as_mut_ptr::<u8>(), 0, 4096);
|
||||
}
|
||||
frames[page_idx] = Some(frame);
|
||||
|
||||
let mut target_flags = region.flags.to_page_flags();
|
||||
if region.flags.contains(VmaFlags::COW) {
|
||||
target_flags.remove(PageTableFlags::WRITABLE);
|
||||
}
|
||||
|
||||
pml4.map_page(page_virt, frame, target_flags);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn unmap_region(&mut self, virt: VirtAddr) -> Result<(), VmError> {
|
||||
let idx = self.find_idx(virt).ok_or(VmError::RegionNotFound)?;
|
||||
let region = self.regions.remove(idx);
|
||||
let pml4 = unsafe { self.pml4_raw() };
|
||||
Self::do_unmap(pml4, ®ion);
|
||||
tlb_flush_asid(self.asid);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Atomically unmap all VMAs associated with `cap_token` (skip PINNED).
|
||||
///
|
||||
/// Hardware access is terminated before this function returns.
|
||||
/// Shared-backed VMAs are unmapped from the PT without freeing frames.
|
||||
pub fn revoke_by_token(&mut self, cap_token: u64) {
|
||||
let pml4 = unsafe { self.pml4_raw() };
|
||||
|
||||
let indices: Vec<usize> = self.regions
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(_, r)| {
|
||||
r.cap_token == cap_token && !r.flags.contains(VmaFlags::PINNED)
|
||||
})
|
||||
.map(|(i, _)| i)
|
||||
.collect();
|
||||
|
||||
for idx in indices.into_iter().rev() {
|
||||
let region = self.regions.remove(idx);
|
||||
Self::do_unmap(pml4, ®ion);
|
||||
}
|
||||
|
||||
tlb_flush_asid(self.asid);
|
||||
}
|
||||
|
||||
//Address translation
|
||||
|
||||
/// Walk the live page table to translate `virt` → physical address.
|
||||
pub fn translate(&self, virt: VirtAddr) -> Option<PhysAddr> {
|
||||
unsafe { (*self.pml4_raw()).translate(virt) }
|
||||
}
|
||||
|
||||
//Activation
|
||||
|
||||
/// Load this address space into the CPU (context switch).
|
||||
///
|
||||
/// Sets CR3 with bit 63 = NOFLUSH to preserve PCID-tagged TLB entries.
|
||||
/// Requires CR4.PCIDE = 1 (enabled by the bootloader/early init).
|
||||
///
|
||||
/// # Safety
|
||||
/// Must only be called from a code path that is identity-mapped in every
|
||||
/// address space (e.g. the kernel `.text` section reachable via HHDM).
|
||||
pub unsafe fn activate(&self) {
|
||||
let cr3 = self.pml4_phys.0 | u64::from(self.asid) | (1u64 << 63);
|
||||
unsafe {
|
||||
core::arch::asm!(
|
||||
"mov cr3, {0}",
|
||||
in(reg) cr3,
|
||||
options(nostack, preserves_flags),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[inline] pub fn regions(&self) -> &[VmaRegion] { &self.regions }
|
||||
#[inline] pub fn region_count(&self) -> usize { self.regions.len() }
|
||||
|
||||
pub fn find_region(&self, addr: VirtAddr) -> Option<&VmaRegion> {
|
||||
self.find_idx(addr).map(|i| &self.regions[i])
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for AddressSpace {
|
||||
fn drop(&mut self) {
|
||||
let pml4 = unsafe { self.pml4_raw() };
|
||||
while let Some(region) = self.regions.pop() {
|
||||
Self::do_unmap(pml4, ®ion);
|
||||
}
|
||||
pmm::free_frame(self.pml4_phys);
|
||||
// Return the ASID to the pool so it can be reused by future processes.
|
||||
// ASID 0 is the kernel sentinel — never freed (guarded inside free_asid).
|
||||
free_asid(self.asid);
|
||||
}
|
||||
}
|
||||
|
||||
// TLB management
|
||||
/// Flush all TLB entries tagged with `asid` (PCID) on the current core.
|
||||
///
|
||||
/// Uses `INVPCID` type-1 (single-context flush) when available (Broadwell+,
|
||||
/// CPUID.07H:EBX[10] = 1). Falls back to a full CR3 reload on older CPUs.
|
||||
///
|
||||
/// **SMP note**: on multi-core systems a TLB-shootdown IPI to all remote cores
|
||||
/// must be added once the LAPIC driver and scheduler are online.
|
||||
pub fn tlb_flush_asid(asid: u16) {
|
||||
// 1. Всегда сбрасываем локальный кэш
|
||||
local_tlb_flush_asid(asid);
|
||||
|
||||
let active_cpus = ACTIVE_CPUS_MASK.load(Ordering::Acquire);
|
||||
let current_core = crate::cpu::lapic::current_core_id();
|
||||
// u64 can only represent cores 0–63. APIC IDs may be >= 64 (CPUID
|
||||
// returns up to 255), so guard the shift to avoid UB (panic in debug,
|
||||
// wrap in release). When SMP with >64 cores is implemented, switch to
|
||||
// a wider mask or a dynamic list of active APIC IDs.
|
||||
let target_mask = if current_core < 64 {
|
||||
active_cpus & !(1u64 << current_core)
|
||||
} else {
|
||||
active_cpus
|
||||
};
|
||||
|
||||
if target_mask == 0 {
|
||||
return;
|
||||
}
|
||||
|
||||
let _guard = SHOOTDOWN_LOCK.lock();
|
||||
|
||||
SHOOTDOWN_ASID.store(asid, Ordering::Release);
|
||||
SHOOTDOWN_ACK.store(0, Ordering::Release);
|
||||
|
||||
crate::cpu::lapic::broadcast_ipi_exclude_self(crate::cpu::interrupts::TLB_SHOOTDOWN_VECTOR);
|
||||
|
||||
while SHOOTDOWN_ACK.load(Ordering::Acquire) & target_mask != target_mask {
|
||||
spin_loop();
|
||||
}
|
||||
}
|
||||
|
||||
/// Full TLB flush (all PCIDs, all addresses) via CR3 reload.
|
||||
pub fn tlb_flush_all() {
|
||||
unsafe {
|
||||
let mut cr3: u64;
|
||||
core::arch::asm!("mov {0}, cr3", out(reg) cr3,
|
||||
options(nostack, preserves_flags));
|
||||
// Writing CR3 without bit 63 forces a full flush.
|
||||
core::arch::asm!("mov cr3, {0}",
|
||||
in(reg) cr3 & !(1u64 << 63),
|
||||
options(nostack, preserves_flags));
|
||||
}
|
||||
}
|
||||
|
||||
pub fn handle_tlb_shootdown_ipi() {
|
||||
let asid = SHOOTDOWN_ASID.load(Ordering::Acquire);
|
||||
|
||||
local_tlb_flush_asid(asid);
|
||||
|
||||
let current_core = crate::cpu::lapic::current_core_id();
|
||||
// Only cores < 64 can ACK in a u64 mask. Cores >= 64 are not
|
||||
// representable; their ACK would wrap and corrupt the mask.
|
||||
// TODO: widen to u128 or use per-core ACK slots for >64 core SMP.
|
||||
if current_core < 64 {
|
||||
SHOOTDOWN_ACK.fetch_or(1u64 << current_core, Ordering::AcqRel);
|
||||
}
|
||||
}
|
||||
|
||||
// Global kernel address space
|
||||
/// The one kernel address space. Initialised once during boot.
|
||||
pub static KERNEL_SPACE: Locked<Option<AddressSpace>> = Locked::new(None);
|
||||
/// Register the already-active PML4 as the kernel address space.
|
||||
///
|
||||
/// ASID 0 = PCID 0 = kernel (no per-process PCID tagging).
|
||||
/// Must be called after `BitmapPMM::init` and `init_cpu_features`.
|
||||
pub fn init_kernel_space(pml4_phys: PhysAddr) {
|
||||
*KERNEL_SPACE.lock() = Some(AddressSpace::from_active(pml4_phys, 0));
|
||||
}
|
||||
133
kernel/src/tty.rs
Normal file
133
kernel/src/tty.rs
Normal file
@@ -0,0 +1,133 @@
|
||||
use core::fmt;
|
||||
use limine::framebuffer::Framebuffer;
|
||||
|
||||
#[derive(Debug)]
|
||||
#[repr(C, packed)]
|
||||
struct Psf2Header {
|
||||
magic: u32,
|
||||
version: u32,
|
||||
header_size: u32,
|
||||
flags: u32,
|
||||
num_glyphs: u32,
|
||||
bytes_per_glyph: u32,
|
||||
height: u32,
|
||||
width: u32,
|
||||
}
|
||||
|
||||
pub struct Console<'a> {
|
||||
framebuffer: &'a Framebuffer<'a>,
|
||||
font: &'static [u8],
|
||||
pub x: usize,
|
||||
pub y: usize,
|
||||
pub fg_color: u32,
|
||||
pub bg_color: u32,
|
||||
}
|
||||
|
||||
impl<'a> Console<'a> {
|
||||
pub fn new(framebuffer: &'a Framebuffer<'a>, font: &'static [u8]) -> Self {
|
||||
Self {
|
||||
framebuffer,
|
||||
font,
|
||||
x: 0,
|
||||
y: 0,
|
||||
fg_color: 0xFFFFFF, // White
|
||||
bg_color: 0x000000, // Black
|
||||
}
|
||||
}
|
||||
|
||||
pub fn set_color(&mut self, fg: u32) {
|
||||
self.fg_color = fg;
|
||||
}
|
||||
|
||||
pub fn clear(&mut self) {
|
||||
let fb = self.framebuffer;
|
||||
unsafe {
|
||||
core::ptr::write_bytes(fb.addr(), 0, (fb.pitch() * fb.height()) as usize);
|
||||
}
|
||||
self.x = 0;
|
||||
self.y = 0;
|
||||
}
|
||||
|
||||
fn header(&self) -> &Psf2Header {
|
||||
unsafe { &*(self.font.as_ptr() as *const Psf2Header) }
|
||||
}
|
||||
|
||||
fn scroll(&mut self) {
|
||||
let header = self.header();
|
||||
let font_height = header.height as usize;
|
||||
let fb = self.framebuffer;
|
||||
let pitch = fb.pitch() as usize;
|
||||
let height = fb.height() as usize;
|
||||
|
||||
let shift = font_height * pitch;
|
||||
let size = pitch * (height - font_height);
|
||||
|
||||
unsafe {
|
||||
let addr = fb.addr();
|
||||
core::ptr::copy(addr.add(shift), addr, size);
|
||||
core::ptr::write_bytes(addr.add(size), 0, shift);
|
||||
}
|
||||
self.y -= font_height;
|
||||
}
|
||||
|
||||
fn draw_glyph(&mut self, glyph_index: u32, x: usize, y: usize) {
|
||||
let header = self.header();
|
||||
let bytes_per_line = (header.width + 7) / 8;
|
||||
let glyph_offset = header.header_size + (glyph_index * header.bytes_per_glyph);
|
||||
|
||||
let fb = self.framebuffer;
|
||||
let fb_pitch = fb.pitch() as usize;
|
||||
let fb_addr = fb.addr();
|
||||
|
||||
for cy in 0..header.height {
|
||||
let glyph_row = self.font[(glyph_offset + cy * bytes_per_line) as usize];
|
||||
for cx in 0..header.width {
|
||||
if (glyph_row & (0x80 >> cx)) != 0 {
|
||||
let offset = ((y + cy as usize) * fb_pitch) + ((x + cx as usize) * 4);
|
||||
unsafe {
|
||||
fb_addr.add(offset).cast::<u32>().write_volatile(self.fg_color);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn write_char(&mut self, c: char) {
|
||||
let (font_width, font_height, num_glyphs) = {
|
||||
let h = self.header();
|
||||
(h.width as usize, h.height as usize, h.num_glyphs)
|
||||
};
|
||||
|
||||
if c == '\n' {
|
||||
self.x = 0;
|
||||
self.y += font_height;
|
||||
} else {
|
||||
if self.x + font_width > self.framebuffer.width() as usize {
|
||||
self.x = 0;
|
||||
self.y += font_height;
|
||||
}
|
||||
|
||||
let glyph_index = if (c as u32) < num_glyphs {
|
||||
c as u32
|
||||
} else {
|
||||
0 // symbol placeholder
|
||||
};
|
||||
|
||||
self.draw_glyph(glyph_index, self.x, self.y);
|
||||
self.x += font_width;
|
||||
}
|
||||
|
||||
if self.y + font_height > self.framebuffer.height() as usize {
|
||||
self.scroll();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::Write for Console<'_> {
|
||||
fn write_str(&mut self, s: &str) -> fmt::Result {
|
||||
for c in s.chars() {
|
||||
self.write_char(c);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
@@ -2,7 +2,7 @@
|
||||
timeout: 3
|
||||
|
||||
# The entry name that will be displayed in the boot menu.
|
||||
/Elyz (RINA KERNEL)
|
||||
/Elyz (LISA KERNEL)
|
||||
# We use the Limine boot protocol.
|
||||
protocol: limine
|
||||
|
||||
|
||||
Reference in New Issue
Block a user