rego: Upgrade to OPA v1#2815
Open
micromaomao wants to merge 24 commits into
Open
Conversation
Contributor
|
@micromaomao - I didn't realize you were working on this. Could we first merge my change and then we can take your updates to move to v1? |
micromaomao
force-pushed
the
tingmao_github/opa
branch
from
c81ddf6 to
d772c5c
Compare
In implementing the fragment parameters feature, we need a way to store, for
each issuer and feed tuple, which parameters should be used for fragments
matching it. Consider this list of fragments, which a parent fragment might
define:
fragments := [
{
"feed": "mcr.microsoft.com/maa/enclavehost",
"issuer": "did:x509:0:sha256:...",
"includes": [
"containers",
"fragments"
],
"minimum_svn": "1",
"parameters": {
"region": "australiacentral2",
"cloud": "Public"
}
},
// ...
]
When we load the fragment containing this, we need to iterate through its
data.<namespace>.fragments array, and for each entry, append the parameters
object to an array that is keyed by the issuer and feed, since we can have
multiple such fragment import entries for the same issuer and feed, but with
different parameters.
This basically means that we need the equivalent of the following code in Rego,
which I've failed to come up with a way to write:
for f in fragment_fragments {
issuer = data.metadata.issuers[f.issuer] or {}
feed = issuer.feeds[f.feed] or {}
feed.parameters = feed.parameters or []
feed.parameters = array_append(feed.parameters, f.parameters)
issuer.feeds[f.feed] = feed
data.metadata.issuers[f.issuer] = issuer
}
While we can dynamically lookup or store based on a key that is from a variable
access, and we can send multiple metadata updates, those updates cannot express
the semantic of "merging" objects or arrays.
To make this simpler, we introduce a new metadata data type - "set", which
supports storing an unordered list of arbitrary objects (which themselves might
be either a string, or an object containing key-value pairs), that can be
inserted to via metadata operations one at a time. This means that we can
simply append to the metadata update operations array once per each fragment
import statement, and the outcome would be the union of all the parameter
objects. This is also very easy to query in Rego, given an issuer and feed:
parameters := [
fp.parameters |
fp = data.metadata.fragment_parameters[_]
fp.issuer == input.issuer
fp.feed == input.feed
]
It is likely that the existing code that extracts included containers/fragments
from a fragment can be simplified by using this feature, but that is outside of
the scope of this PR.
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
…and improve comments Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
…ching strategies This is to make it easier to parameterize environment rules. Currently, name and value for an environment rule are actually combined into one "pattern" field, and there is only one strategy for the combined pattern. This presents a problem when a fragment wants to delegate the decision of e.g. whether to match the value (but only the value, not the key) with a regex or with a fixed string. We split "pattern" and "strategy" out into "name", "name_strategy", "value" and "value_strategy" in order to allow more flexibility when fragment exposes env-var parameters. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
Also fix wrong assertDecisionJSONContains - the original code for Test_Rego_EnforceEnvironmentVariablePolicy_NotAllMatches(_Windows)? only asserts that a key that is valid exists in the decision JSON, but this is incorrect because it should had verified that the decision JSON mentions the invalid key. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
This commit implements the support for the "parameters" feature for fragments.
This is achieved by having the framework extract the parameters object from
fragment import statements, and storing them in a set, tagged with the
applicable (issuer, feed) pair. On future fragment loads, if the fragment's
issuer and feed pair matches with any previous entry from this set, the
parameters will be passed to the fragment via an automatically injected object
added to the end of the fragment's Rego source (__fragment_parameters).
(Note that in reality, we need to combine this set with the import statements
defined in the main policy. c.f. candidate_fragments. This is done in
fragment_parameters_for)
In order for fragments to use this parameter object, it is expected that all
fragments will now have an additional "stub" inserted at runtime to define the
parameter() function, which will, under the hood, reference a "hidden" variable
__fragment_parameters, also inserted dynamically at runtime, containing the
actual parameter values.
If multiple fragment parameters are specified, all combinations of values are
considered "allowed", and therefore the fragment injection is repeated, passing
in different parameter combinations, one for each combination. This means that
if a fragment defines, for example:
containers := [
{
...
"env_rules": [
{
"name": "SOME_KEY",
"name_strategy": "string",
"value": parameter("my_param"),
"value_strategy": "string"
}
]
}
]
and we have two fragment import statement for this fragment, e.g.:
fragments := [
{
"feed": ...,
"issuer": ...,
"minimum_svn": ...,
"parameters": {
"my_param": "value1"
}
},
{
"feed": ...,
"issuer": ...,
"minimum_svn": ...,
"parameters": {
"my_param": "value2"
}
}
]
Then the container can be started with either SOME_KEY=value1 or
SOME_KEY=value2.
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
---
Changes:
- Fix missing [_]
- fix wrong usage of defer
- Inject fragment parameter function definitions at runtime:
We must inject this at runtime so as to avoid having to support this exact
implementation (eg __fragment_parameters[name]) of fetching the parameters
forever (as it will essentially be hard-coded in the generated policy).
- Move the actual implementation of parameter() into the framework so that
fragments doesn't have to say 'import future.keywords.every', 'import
future.keywords.in'.
- Use indirection with default when accessing fragment.parameters to not break
older fragments.
If we don't do this, loading a fragment without the parameters definition
would fail due to "unsafe" rego variable references.
- Fix broken tests caused by the new extract_parameter framework function
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
…ing env list If every element in the input env list matches some env rule in some container, but there is no container that matches the entire env list, we currently deny correctly but returns no error message. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
This will test for scenarios like different fragments using the same parameter name, multiple parameter combinations, correct parameter passing for nested fragments, and make sure container creation is denied if the parameter and the given environment / command doesn't match. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
Name still provisional, might change later. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
…ching strategies This is to make it easier to parameterize environment rules. Currently, name and value for an environment rule are actually combined into one "pattern" field, and there is only one strategy for the combined pattern. This presents a problem when a fragment wants to delegate the decision of e.g. whether to match the value (but only the value, not the key) with a regex or with a fixed string. We split "pattern" and "strategy" out into "name", "name_strategy", "value" and "value_strategy" in order to allow more flexibility when fragment exposes env-var parameters. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
Also fix wrong assertDecisionJSONContains - the original code for Test_Rego_EnforceEnvironmentVariablePolicy_NotAllMatches(_Windows)? only asserts that a key that is valid exists in the decision JSON, but this is incorrect because it should had verified that the decision JSON mentions the invalid key. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
…ired rules present This refactors rule_ok (and renames it) to fix the `some env in envList` being applied at the wrong level. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
This currently support containers[_].env_rules and containers[_].mounts. If multiple platform_rules are defined, a container matching either one can be started (but in a consistent manner - e.g. if two platforms have different environment variables or mounts, a container can't "mix and match" between them). In order to achieve the above consistency, we "patch" the container objects instead of adding logic to e.g. env_rule_ok or envList_ok. This also means that the error_objects of a denial message will reflect the platform rules inserted. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
…e field Currently we only add input.rule to the original input, not the redacted input. This results in the case of create_container not having the "rule" field in the final deny message, but other enforcement points do have it since in those cases the redactSensitiveData return the original input map. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
# Conflicts: # pkg/securitypolicy/framework.rego # pkg/securitypolicy/regopolicy_linux_test.go
Revert some "rego.SetRegoVersion(ast.RegoV0)"
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
go: upgraded github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 => v4.4.1 go: upgraded github.com/goccy/go-json v0.10.3 => v0.10.6 go: upgraded github.com/lestrrat-go/blackmagic v1.0.2 => v1.0.4 go: upgraded github.com/open-policy-agent/opa v1.4.0 => v1.18.2 go: upgraded github.com/prometheus/common v0.66.1 => v0.67.5 go: upgraded github.com/prometheus/procfs v0.16.1 => v0.20.1 go: upgraded github.com/rcrowley/go-metrics v0.0.0-20200313005456-10cdbea86bc0 => v0.0.0-20250401214520-65e299d6c5c9 go: upgraded go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 => v0.69.0 go: upgraded go.opentelemetry.io/otel v1.43.0 => v1.44.0 go: upgraded go.opentelemetry.io/otel/metric v1.43.0 => v1.44.0 go: upgraded go.opentelemetry.io/otel/sdk v1.43.0 => v1.44.0 go: upgraded go.opentelemetry.io/otel/trace v1.43.0 => v1.44.0 go: upgraded google.golang.org/genproto/googleapis/api v0.0.0-20260414002931-afd174a4e478 => v0.0.0-20260526163538-3dc84a4a5aaa go: upgraded google.golang.org/genproto/googleapis/rpc v0.0.0-20260414002931-afd174a4e478 => v0.0.0-20260526163538-3dc84a4a5aaa Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
Replacing
(^\w+(\(.+\)|\[(\w+|".+")\])?( := (\w+?|\{(.|\n)+?\}|\w+\[\w+\]))?) \{
with
$1 if {
micromaomao
force-pushed
the
tingmao_github/opa
branch
from
d772c5c to
9d58ac7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends-on: #2789
Depends-on: #2792